Conversations- with not very intelligent machines

Over this incredibly enlightening week, there were a few breakthroughs and a get back to the drawing board moment. My earlier assumption that Human Speech and technology coming together will revolutionize our interactions, was changed in a bigger way. I am unsure if this is good or bad, fully aware that the scope of my thinking has changed drastically.

Frantically researching over the course of the week has lead me to revisit the two approaches to technology: AI-NLP and AI-MP. Naturally this leads to digging deeper in the AI world and understand fundamentally where things lie. This is going to be a longer research project than the last few weeks have been.

No time to waste, when there is so much to do. 

Making Good Voice Interface

This week has been very informational with excellent conversations with professionals developing voice based applications. As an outsider, all the conversations started in the tentative mode. I need to get some insights and knowledge from the professionals in the world and not irreparably damage the relationship. The next objective was to get a buy-in for validation and if possible get a on-the-spot validation of my understanding of the space.

The part where we discussed the potential of the space was unanimous. Voice is seen as having the potential to change the way we interact with computers- no doubts about that. The momentum in the market is rapidly accelerating without a doubt. Alexa and Echo were truly the culprits that opened the floodgates to bring the innovations in voice to a wider audience attention. All this was very good news!

But

There had to be a catch. The development and design using voice was struggling with the mechanics of designing great user experiences. There were a lack of tools and the investments in using the tools to make voice work is so large that most product owners focussed very little time in revisiting the users. They were largely content in morphing the current controls to be activated by voice. 

A typical example of the capability of voice in a car navigational system is when a driver is able to get the results of a multi-step point and click workflow completed with one request. The perfect value-of voice can be seen when a driver asks for "a 4 star or above rated Asian restaurant  within 10 minutes"; and the system is able to parse out that this is a search for a restaurant around the current location, calculating for speed, traffic etc and filtering for cuisine and ratings to provide the most efficient options under the circumstances.  

A good VUI design would take the means to the ends approach, whereas the "adapting controls to voice" approach would envision three or four back and forths. Understanding the point of interaction - find a decent place to eat and then solving for that need is the key to a successful VUI design. An approach not taken by VUI designers today.

Most product owners are content to make "voice" as one more "control" to navigate the clunky and intellectually dumbed down world of point and click. I have seen and read several lab experiments with primates on behavior- which use a very similar format across the board. A primate needs to accomplish a set of tasks starting with some trigger and following a pre-set sequence to be rewarded a treat. Most of the voice controls feel like sounds being the trigger and the tasks saying the same.

Product owners developing in the voice realm have to understand that with Voice, you have replaced the "ape" with a "human" and the experiment is over. You have to learn new things about motivating your subject and device new mechanics to deliver the treat. Failure to do so will turn your "human" audience into "apes" and they don't like that at all. You just end up causing a lot of frustration.

Conversation Design

The whole purpose of Conversation Design was to recognize the intellect in your users and provide you with tools to design for that higher level of intellect.

Voice Systems Recap

In the past three months that I have been tracking Alexa, the numbers of skills in the Alexa store have been growing from 1600 in November,  4000 in January to 8700 today February 9th 2017. The growth in the skills is doubling in less than a month.  Considering that between June 1055 and 2016 they had experienced a 0-1400 in a year and added a mere 200 in the following 5 months.

Amazon Alexa Skills Growth

This signifies that the ecosystem is booming. The quality of the skills available is also improving significantly over the last two months. Based on the capabilities available to developers, skills today have significantly better abilities than just a few months back.
Over the past month I have met two types of people, those that have used Alexa can understand did development in the voice technologies and those that haven't. Folks that on Alexa and use it regularly understand the capabilities and utility off a voice interface. Folks that don't use Alexa have experience with Siri and Cortana, not really reflecting the true development in voice technologies.

The voice architecture

Based on my research so far, I have come to conclusion that the voice technology components can be summarized into the following.

The Voice Architecture

1. The human interface device: this is the primary mechanism by the which a user interaction with the system. For Amazon and Alexa this comes in the form off an Echo speaker or an AVS enabled system from one of the hardware Vendors partnering with Amazon.
2. The speech to text module: this is the component that has achieve significant breakthroughs in the past couple of years. Microsoft last week announced deck the speech to text technology is on par with humans for dictation.
3. Text to speech module: this compliments the speech to text in converting return text into speech. Most off the voice systems today support speech synthesis markup language that provides control over how the human interface device when does the text as speech.
4. Natural language processing System: this consists off two separate modules the Automatic Speech Recognition (ASR) module and the Natural Language Understanding (NLU) modules. These work in tandem to recognize the context as well as the intent of speech by the user so that the system can react appropriately.
5. The next set of modules signified as the Switchboard, the Voice Interface and the Apps sit behind this front-end voice stack and generate conversational content in the right domain.

All of the different vendors provide different interfaces for applications to leverage the voice interface, the mechanics of using the stack is quite similar. Users utterance is captured by the human interface device processed by speech to text and NLU to identify intent we just passed onto the right application for processing. The application is responsible for handling the request and generating the response which is conveyed back to the user through the human interface device.

There has been significant advances in the entire stack that makes the system viable and usable with a relatively moderate amount of effort. My previous posts have attempted to journal the research and try to validate my thoughts and my hypothesis that Conversation design requires a new approach to user interface. This appears to be on the right track #conversationdesign.

The alarm-a personal experience with Voice.

Every day I set the alarm using Alexa the command is straightforward and almost the same every day.

"Alexa wake me up tomorrow at 5:30 AM".

To this command Alexa responds as "alarm set for 5:30 AM tomorrow"

I have been hearing the same response for the last month. In a well designed conversation this interaction could go differently, in several ways. Alexa could go with one of the following options.

"OK " Or "OK I'll wake you up at 5:30 AM" Or "You got it" Or  " 5:30 am tomorrow".

But Alexa does not do this. I hear the same responses over and over again every day. If I forget to say a.m.  Or p.m, Alexa immediately prompts me the next question "is that in the morning ? or in the evening?". This interaction has a promise of a great conversation but the overall experience falls short slightly.

Natural Conversation

How do you go from an artificial or synthetic conversation to a more lively one?

You mix things up a bit, just a bit so that there is some personality to the interaction. Every user needs the assurance, the feel that the interaction is going in the right direction. Progress has to be made without being repetitive or boring or synthetic. The flow has to be as natural as with a human.

Working with Alexa every day, leaves me wanting more just a little bit more. I really want to have a conversation not in a personal sense but in the same way I have one with other humans. This is not a huge ask. It merely requires a little more effort on the part off of the UI designer. A designer just needs to add a few variations, sprinkle in some nuances and keep a memory of how things worked so that the interactions improve, become more natural and continue to build confidence between the user and technology.

Alarms are typical example of how people do things over and over again. I have two different times of alarms during the weekdays.  Technologically, it is not hard for Alexa to remember that I wake up at 5:30 AM or 6:30 AM depending upon the day of the week. I do not have to specify exactly the time I need to be woken up every 24 hours or 25 or 23 hours.

Alexa does not yet have a regular alarm setting. Are you still me a set an alarm only in the next 24 hours from the time it is set. This means our user has to bear with the same monotonous and boring responses from Alexa each and every day. Even though Alexa Voice Service has several best practices and guides to program into the voice system I really get to experience no variations in the daily routine interaction.

90's Technology

This experience is similar to the time when I was programming the mouse click events without following any standard guidelines. My programs did what I  developer #n preferred, my colleagues programs did what they preferred or thought was much more intuitive. Our model was intuitive to us the developers but we created a nightmare for the users; or more work for one of us later when we had to snap to the same format after Q/A caught up with us. We would have passionate debate on the best way to click the mouse in an application. Just within one company, each feature team implemented the interface based on their preferences and the winner of the internal debates. As an industry the users suffered greatly until all of this was resolved with the standardization of the mouse interface.

What happens when a user right clicks, left clicks, click and drags, Double Clicks etc was standardized, in   pre-defined ways. The mystery and the mystique of programming the mouse was gone. And everybody was able to use the computer equally well irrespective of the program or application.

The Solution

Will conversation design get there? Definitely, it is far from there at this moment in time. We all technologists, UI designers and fellow-humans need to work together to bring about some standardization so that we all benefit from each others learnings.

This is my first ask for comments on variations you have experienced in daily routine conversations. Please send me your experiences and the impact in terms of freshness or déjà vu to help educate the user experience with voice.

PS. This was the first blog or piece of content drafted using voice dictation. It’s a pain so far but less than typing it up.