The Visualizer app from AkzoNobel, is a free Augmented Reality tool for iOS and Android, which enables consumers to re-colour their walls while viewing the results live on their mobile device, as they move around the room.
Choosing colour for the home can be a stressful experience, and in the UK alone it is estimated that 3m households a year give up on painting projects at a cost of over £200m. By providing a tool to quickly visualise paint colour, in-situ and in real-time, consumers can quickly and confidently choose the right colours for their home, prior to using paint testers.
For AkzoNobel, the app also promises to significantly shorten the sales cycle through e-commerce integration, while strengthening their global brand portfolio, encompassing some of the world's best known paint brands such as Dulux, Flexa, Nordsjö and Coral.
The Visualizer was launched globally throughout June and July 2014, in over 40 markets, in 18+ languages and across 25 brands.
Development of the Visualizer
In 2013, AkzoNobel partnered with String® on what was initially an R&D project, to explore the possibility of a computer vision based broad wall visualisation application. Computer vision is a discipline within the field of computer science, which focuses on processing and applying visual data from the real world, in a similar way to how our own eyes and brains work.
Handling the diverse range of room environments and lighting conditions which may be encountered, presented a number of technical challenges. In order to overcome these, String® researched and developed proprietary algorithms bespoke to the Visualizer, to analyse and process live video from the device camera.
These algorithms identify broad areas of interest - in this case, walls - and then isolate household objects, such as TVs, picture frames and furniture, to prevent them from being recoloured.
Ambient lighting conditions in the room are measured, so that when the ‘paint' is applied, shadows and subtle variations are retained for more natural looking results. Finally in order for the virtual ‘paint' to remain locked to the wall surface as the user moves around, custom software was written to process positioning and orientation data, from the device's hardware sensors.
Key Technical Requirements
- CV based solution, rather than relying on time-of-flight or FLIR sensors, due to the current saturation of traditional camera-based devices in the market.
- Automatically isolate features such as windows, doors, TVs and paintings when applying virtual paint to existing walls.
- Process and analyse the real-time ambient lighting conditions to ensure natural looking results when re-colouring, retaining surface shadows and subtle variations.
- Track the environment and user inputs to remember walls that have previously been painted during that session.
- Provide an immersive experience for the consumer with high frame rates across a broad range of consumer tablet and smartphone devices.
- Deliver as a cross platform SDK for integration into the broader Visualizer application.
User Experience Design
Due to the amount of screen space required for the Visualizer's camera feed, one of the key design objectives was to keep on-screen clutter to an absolute minimum, to avoid hindering the fluidity of the user's journey, while providing all the tools they need.
Leveraging our commercial experience in interaction design, we built an in-house custom prototyping framework which allowed the team to rapidly design, iterate and test interaction concepts from desktop to mobile, prior to final API development.
Some of the concepts the team explored included accelerometer-based gestures for rapid clearing of paint within a scene, simple touch gestures to refine colours and swipe-able colour palettes, in addition to standard touch events.
While many ideas were discarded in order to reduce user friction, key features such as single tap-to-paint, visual paint anchors (with inertial fade-outs), masking tape, inclusion of the colour swatch on screen grab, and low light notifications, all emerged from this process and were implemented in the release.
Our belief is that technology should work like magic: providing a delightful, easy and useful experience. This ethos remains at the core of all of String's technical and creative endeavours.
The uncompromising vision for the project required String® to break new ground in this field and what you see today represents a first step towards that vision. On-going research and development will bring significant advances to future versions of the product.