With the Christmas shopping period rapidly approaching, Amazon Echo products are likely to be a top item on this year’s gift lists for those eager to explore the opportunities of a digital home assistant. As the Alexa – Amazon’s intelligent personal assistant – product range extends, it has also become a key platform for several third-party products. Increasing pressure is mounting on these third-party vendors, including Amazon, to deliver seamless user experiences.

The Internet of Things offers increasingly sophisticated capabilities, and the opportunity to transform our everyday lives. In turn, demands have grown exponentially around user experience and product quality. Today, IoT systems are highly distributed and diverse – to ensure everything runs smoothly, including graphically rich systems, testing is essential.

Increasingly complex user interfaces

The Echo portfolio is a key example of how we no longer simply interact with digital devices via a controlled environment such as a keyboard, and user interfaces are no longer ‘just’ digital forms for us to navigate, consisting of components such as labels and buttons.

Through services including Spotify and smart home devices, voice and video are layered in: this requires testing the entire system end-to-end to ensure delivery of a seamless user experience every time. Testers are no longer saying something is ‘out of scope’ simply because it was not developed internally. Ongoing monitoring of user experience is now a key requirement for success in a digitalised era.

The importance of multi-level testing

When it comes to voice, testers need to be able to ensure a great user experience at each point of interaction—including how Alexa handles different regional accents and creates appropriate responses. Failing to do so can cause end-user frustration – and an embarrassing situation for Amazon if anecdotes surrounding mishearing home assistants go viral.

As vendors add third-party capabilities into ecosystems, they need to be able to test (interact/validate) the entire end-to-end experience, involving all components, at every level. For example, if you have multiple user interfaces (your Alexa voice-activated UI, phone, laptop, and fridge). In addition, you have multiple IoT systems in your house gathering information from lots of different sources, such as a power meter gathering information from lights, fridge, and heating. So, you have several things producing data, several things aggregating and processing data, and several things presenting data. As a result, testing becomes increasingly about testing a set of independent, technologically different components working together to do something.

Setting the (user experience) standard

If businesses want to gain a competitive advantage in the digital personal assistant market, they need to consider implementing a set of standards: for example, the assistant must be able to respond within a certain amount of time, work with simplified English, deal with different accents and provide helpful error messages. These should be met from an end user experience point of view before other service providers can become part of the Echo ecosystem, and must enable those providers to easily achieve the same quality of user experience.

The ability of users to smoothly and accurately interact with any IoT solution will decide whether it succeeds in the over-saturated landscape. If Amazon wants the Echo to be an essential tool for years to come, and not just for Christmas, then it needs to guarantee an experience that delights all users – at all times.

Written by Antony Edward, CTO of Testplant