Software testing considerations for voicefirst applications

The names Alexa and Echo are household names and someday soon, most people will have these devices in their homes, ordering takeout, picking out a song, answering trivia questions. Welcome to the voice-first applications era. Amazon and Google have sold millions of Amazon Alexa and Echo and Google Home devices and 24.5 million voice-first devices are expected to ship before the end of 2017 Gartner is making the call that worldwide spending on virtual personal assistant (VPA)-enabled wireless speakers will be over $2 billion in the next three years.

A voice-first device is a smart device specifically designed to get tasks done conversationally. They're always-on and always-listening. Businesses must be thinking about how they can leverage this new channel to provide better service for their customers. In the same vein, developers and designers are thinking about how they can design and deliver the best voice experience. And unfortunately, we know that hackers are already able to break into these types of devices. Security specialists need to think about securing data and network access in the event of a hack attack.

There is a specific set of challenges in executing software testing for voice-based apps like Alexa skill or Google Home actions where software testers find bugs before the customers do. Testers automate testing processes so updates can release faster and with higher quality. To keep up, we need to adapt processes to the new technology.

An effective and thorough test strategy starts with a good understanding about the underlying architecture of the system under test. The tester needs to understand the command details from the moment the user starts asking. For instance, the voice first device is asked to do something until the task is complete. In this case, the user gives a command by saying the wake-word 'Alexa' and follows it with command details. The device sends the voice stream to Alexa and its does a couple things, using speech-to-text technology to translate the voice stream to text and identifies the skill name and maps the user's spoken inputs to the skill's intents and slots.

It's important to note that there are many different types of testing  -  unit, system, integration, performance, endurance-  they all carry over to voice, but there are challenges, such as standard testing is text-based.

The Automated Voice Testing (AVT) approach use a speech recognition engine to automatically control the device conversationally as a human user would. This approach is device-agnostic and platform-agnostic because the automation of the tests is at the voice level. The automated tests can be executed against any kind of voice app, running on any device.

Some use cases where AVT can be applied include Alexa app testing (Amazon Echo device) and Google Home apps testing (Google Home device), as well as voice-enabled web sites and web apps, virtual assistant testing: Siri, Cortana, OK Google on Android, and automated communication: call center, voicemail, etc.

Voice-first devices have gone mainstream and are providing a new way for businesses and brands to better engage with their customers by delivering a fast, easy, and familiar voice experience. Testing the software that powers the innovation is up to the task, helping users feel more confident about their experience with this burgeoning technology.