Eliminate the queue with Voice

EXCITING DAY, this last day of January 2017!

Starbucks announces a "reorder skill" for Amazon Alexa. This is a huge step in disruptions through voice. The mechanics of ordering a coffee may sound mundane and not very exciting as we have been doing that before we invented technology. But, this is a huge step towards the technology development. This is the first step in testing the business models powered by voice.

Now, I am not a regular Starbucks customer but found myself creating an account just to try out the conversational experience. More importantly, follow the evolution of the conversational experience as the skill evolves.

It is amazing to see the skill being rated so high for a first version. Cant wait to play with it some more and write about it.

https://www.amazon.com/Starbucks-Coffee-Co/dp/B01F9RRL46

Folks at Starbucks, you should checkout the #ConversationDesign book for tips on evolving the conversation from this initial version. I have a Pizza pick-up as an example but the result seems to be similar.

The days are the voice are definitely near.

The Pace of Innovation

Today, I came across the WSJ article on LinkedIn about the "complications" a echo box brings to a household with someone that has a name similar to the wake word. I am pretty sure someone at Amazon did a lot of research to come up with "Alexa, Amazon and Echo" as wake words and later added "Computer". I am curious to the probability of a household that has names that sound similar to all these four.

Jokes aside, I had noticed a new behavior with Alexa. A month or so back, anything that sounded like "Alexa" got us a voice prompt or a response, whichever way you take it. 

"I'm sorry, I did not understand the question" or something along those lines. This is precisely the behavior that  is written about in the article. The need for the Alexa Voice Service (AVS) to do something at the expense of disrupting the ongoing activity.

The Change

This behavior has changed, quietly and while we were not looking. The power of designing the service in the cloud showing through. In the past couple of instances, we were talking about Alexa, using the name several times. This definitely woke up the system with the blue green light ring dancing in excitement just like a puppy ready to jump at you for a conversation. But, this did not happen, the box stayed silent when it realized the utterances did not merit a engagement.

The implications of this change in behavior is huge. the article was outdated almost immediately. The challenges, the factors, the shortcomings raised in the article are moot within days of them catching the fancy of the media. A fascinating development in pace of innovation int he industry. We had heard that businesses need agility to respond to the needs of the market and customers, this demonstrated how it was done.

Take a bow Amazon, I am impressed.

The Power of Silence

The more important development in this iteration however, is the fact that AVS has graduated from the phase of an eager untrained puppy-that is annoying and an embarrassment to pet parents to control; to a well behaved, cute and adorable creature. The energy, enthusiasm and eagerness is contained in an attentive silence. I believe this is a huge improvement from Alexa in December to January. This is dog-year development we are talking about.

Silence as a tool of communication is well written and even advanced, mature and wise humans find it hard to employ in the conversation. An electronic system has beaten us to it. Hats off to the designers and product owners that prioritized this over the gazillion asks coming from customers and partners.

#ConversationDesign now has a very powerful tool to employ in maintaining credibility with your user. Silence. This merits a new chapter in the book.


  

The ways of humans

Source Magnification Effect - the paradigm that an individual is more convinced by an opinion or a statement when it is expressed by a variety of people than by one person; is the dominant principle at work in our digitally connected always on world.  The last two months have provided me with ample opportunities to experience it first hand. Let me explain:

After playing with Amazon Echo the last week of November, I realized the opportunities in Voice User Interface. As any disruptive idea, I approached the subject tentatively with my co-workers, colleagues and confidantes. By the 20th or so conversation, the tentative nature of the argument was replaced by energy- unlike anything I had experienced in recent years. By the first week of January, the enthusiasm, the twinkle and passion was uncontained. I dare you, bring up "Voice" and you will be drenched in enthusiasm and energy.

That is a tall order for a gentle, calm and collected individual with few words. Learnings leading to notes; notes to blogs; blogs to articles; articles to books and published book being purchased with absolutely no promotion. That is hitting a jackpot.  The story does not end there just yet. In the last week alone, I have seen article after article validating my the initial impressions. There is not just validation but information, clarity and details that seemingly incorrect articles and whitepapers published on slideshare being rectified along the way. Wisdom and knowledge definitely have their own way in finding th motivated.

Weather it is an Amazon employee showing off the Echo to a tech audience for the first time, with all the errors and shortcomings; or it is a Google VUI guru explaining my thoughts - Verbatim. The momentum of my, nay our line of thinking is just beginning to gather substance and take much defined shape. We might disagree on the approach and paths, but we all agree that the technology today works well enough to be put to good use.

There is value in exploiting voice as an input-output mechanism; even in its imperfect form. Imperfect form might be the best reason to get it to work for us.  I was pleasantly surprised to see the imperfections and approaches articulated somewhat along the lines of my thoughts- though not completely the same

1. We thought we knew how we converse- we don’t
2. We thought we could teach machines the science of conversation- we cant do it just yet
3. We thought conversations need perfect information- we are not sure about that.

If you are looking to change the world, this is your opportunity. I strongly believe that when we take on this challenge of teaching machines how to work with us; we may learn a thing or two about ourselves. Source Magnification Effect is just the beginning.

I am just beginning to experience the Informational Social Influence effect. The effect where several people share my belief even though we did not collaborate, corroborate, connect with each other or rip off each other. We all followed independent investigations for different purposes, generating information and processing them for our own purposes; yet we reach the same conclusion. The conclusion is definitely the same. Voice is going to change the world.

Voice brings the next opportunity. Voice is a large undertaking and will take over a decade to mature into a commodity. There is a lot of work needed to get there. Our mission is clear, we teach the machines the ways of the humans.

#ConversationDesign: teaching machines to work with humans. Naturally.

Assessing the voice technologies

Even though we had several iterations of voice activated systems over the years, technology that would in some way mimic conversations and replace the venerable mouse and keyboard seemed to be far into the future.  As a consumer and user of technology, I have a vested interest in voice and wanted to be crisp and clear in articulating my asks

• I hate to type. Having had no formal training in the typing skills, I am slow, inefficient and error prone. Using a finger closest to the key to be pressed strategy, keyboards are becoming painful, just as I get to a point where the amount of typing is steadily increasing.
• As a human, I have several input and output faculties. Out of the five: vision, touch, sound, voice, smell; the drop off between sound and voice is significant. A more balances approach is definitely more desirable
• Over the past few years, I have begun to rely on the track-pad and touch sensitive screen quite a bit leading to painful hands. Trying to get things done with the unnatural systems is taking a toll.
• I want the ability to choose the most preferred mode of using technology. The choice should be mine and mine alone, without any rigidity. I do not want one natural system replaced by another one.

As the technology evolves and gets better, there are definitely risks to my consumer vision. Based on my research of the voice systems here are some visions of the big players that have the potential of causing more unnatural interactions for me. Let us stat with the top players and their mental models.  The following section is my view based on the approach, system design and actions and less on the words in the marketing.

Amazon

Amazon believes that most people have no need for the keyboard or mouse or even the computer. If everything you need to be done can be done in the cloud, why would you need a computer? You just need a speaker/microphone combination that lets you communicate, learn about things, get entertained or complete tasks all through a voice interface. You wont need a computer. All you have to do is to Ask Alexa. Even today with the constructs like "Alexa, ask the door to open", Amazon is programming the users to work with this imperfect technology

Google

Google believes that  the biggest problem in the world is information overload and with a voice interactive systems, users will be able to get to the information better. I feel Google is completely in the discoverability followed by the monetizable conclusion of the interaction. In simple terms, Users don’t need to remember anything, if they need something to be done, google will help you find who can help and help you get it done. 

Microsoft

Microsoft as always an interesting company.. Do it all. With one of the least sophisticated ASR and NLP systems, they cannot seem to decide weather to take the right road for consumers and protect the Windows business. Build a compact, portable ASR+NLP+TTS system, pack it into a home computer- or a newer iteration of it or follow Amazon with a "Service" model and run it in the cloud. There is of course the green grass on the Google side as well. This is unfortunate considering the fact that they are best suited to serve my requirements.

Apple

With Siri, Apple sees a way to stay relevant with the consumer devices based on iOS, Mac and tv countering the strategy followed by Amazon.  Apple sees no reason why a balanced use of senses cannot be made to work for the users. The interfaces can be a bit better but the vision is in the direction to my liking.

Others

There are several other players in the market without the ASR and NLP engines and hence using one of those provided by the big four. While some leverage the capabilities provided by the big four, others are building a fully integrated stack specialized for their customers. Just like Garmin or Tomtom.

The Open source community is not far behind with STT and TTS engines, IFTTT technologies and other components organically popping up around the world. I am very optimistic about the open source world and hope to join in the movement soon.

There is a place for everyone in this space and the chess boards are just being setup.

Keyboard to Mouse to Voice

I know a lot of people that have been around to appreciate the evolution of Input and Output devices through the evolution of the computer industry. I was lucky enough to work on the 8080 controller that had a keypad and programmed using assembly language to simulate a  lift. One of my first programs that was a challenge in my class to build using the minimum number of CPU cycles. I believe the contest was a draw between me and a few other classmates for the grand prize of a Cadbury's éclair.

Next came the computers with keyboards without a mouse. A world that required a deep knowledge of commands, shortcuts and working on a scrolling screen with applications on several separate floppy disks.  The mouse brought a new level of luxury, you did not have to remember the shortcut to a badly designed menu item, where the developer forgot to underline the shortcut key; click on it and get to the sub-items and execute right away.

I remember programming  mouse-down, mouse-up  events, then later click and doubleclick events, followed by classes that abstracted it out. All this was gone with WYSIWYG interface and developers did not have to look under the covers. The amount of effort over the five odd years to get mouse to be invisible- in a sense, something you don’t think about; was incredible.

Many of my friends consider the touch sensitive screens as the next innovation, to me, it was just a combining of screen and mouse. The effort from going from "raw processing" of events to standardized swipe, pinch etc. took about five years.

 

Voice

I believe with the developments in voice-to-text and text-to-voice along with computers today, Voice is the next leap after the venerable mouse.  This may be a disputed statement, and many people have pointed out the various limitations, the errors and the inabilities and the chunkiness of using  voice. It is true that when you think of voice as a channel of communication, there are many things to be desired, but if we look at the voice as the next I/O channel, the world around us changes.

Let me put it another way: with keyboards, mouse and a combination of the three, we- the humans learnt the skills to operate this machine we call computers.  These computers have capabilities due to software running on them. With Voice, the machines have to be taught to work with us; Machines bear equal responsibility in the interaction. Given the state of the technology today, that is a tall order to fulfil.

Conversation

Humans know conversations inherently, it is a fundamental skill mastered early on in life with varying levels of competence. Every human has vast reserves of information, experiences and strategies to navigate a simple exchange between two people. We are attuned to various cues even without visual feedback that we consider in our exchange effortlessly.  We do not yet fully understand all the aspects of how we converse to be able to model machines to mimic.

A huge body of researchers, scientists and engineers believe that information, precision in interpretation and algorithms will enable computers to have voice conversations with us. Yes, I believe this is true. It is not a question of if but when this will happen.

Wait or Act

Do we wait for the technology to develop where machines can have conversations? Or do we go ahead and program conversations with the technology we have at the moment? I believe in the latter approach. We have to start now and program with the limited technology and get going right away. This approach will enable us to push the envelope of our conversational understanding- the mechanics beyond content.

Starting now, will get us on the co-evolutionary path where we will get familiar with the system, the interface, the design, the users and build a body of knowledge that will enable us to make hard choices when voice is as ubiquitous as the mouse.

Passion for a voice

This week while I was having a conversation of Amazon Echo over beer, someone mentioned the reluctance in adopting the technology at this time.  "I want to have a conversation with Alexa. Maybe, even give her a name and voice that I can fall in love with. Just like I love my vintage car". This was not the Big Bang Theory kind of fall in love with Siri, it was admiration and passion for the product that worked with an intelligent human being with a voice.

The profundity of this statement pushed the boundary of  the voice opportunity, much further than my earlier realization that, people want technology to work for them. Not in a " I know it all" kind of snobbish way, but genuinely helpful technology.  It is not the efficiency of how the task is achieved, but the manner in which machines put in the effort to gracefully and patiently interact with you to get you the information, accomplish the tasks or just plainly chat with you. It is not intelligence that people look for, they look for normal.

Artificial Intelligence

The concept where the machines will be smart enough to work with us using speech has several dimensions. In my research on voice, several books point to one simple fact. We do not completely and accurately know how we-humans acquire conversation skills. We start the process as babies and by the time we can experiment non-intrusively, it is too late. Just the thought of any scientist experimenting with babies would be repulsive to anyone. Therefore we cannot model the machines based on us, we don’t know how we do it.

Designing for Intelligence

There is a very large effort behind developing intelligence, building models, trying and testing them out to convert data into intelligence. We are going for a world where machines can have intelligent conversations with us.

I believe that the technology today exists that can be leveraged to design for intelligence within the limits of current capabilities. At the very least, we can get a better understanding into how we converse, under various real circumstances. We can understand what is "Normal" better.

Infant Voice

When we talk to infants, we use simple bite sized chunks of words, phrases even to interact.  We do so in thousands of scenarios within the first year of the baby's life. Each of this gives the baby the skills, tools and data to start with words and graduate to phrases- mimicking adults in a matter of months.  I believe we are in this stage of the development of the voice technology.

We would work with voice better if the voice was that of an infant. One that is eager to do things for you and please you - just for the opportunity to interact with you. An infant, that would make mistakes and understands little but whatever he/she understands, is able to act on with gusto. Who would not think this is cute? This would perfectly align with what current skills for voice systems.

Not everyone would like a child's voice but maybe- just maybe we can set the right expectations with the users. Clearly, this is not available at this time.

Child's Voice

As we learn more, develop more capabilities, we can frame simple sentences. Crisp, clear, concise and sometimes insensitive - this is what we are designing current voice interactions today.

"Alexa, bring me my paper."

"Hmm, I 'm not sure what you meant by that question?"

Responding to an ask beyond ability should be an awkward situation and not something graceful. We do it all the time. To me this response from Alexa is not a technology limitation, it is a VUI gap.  A child could respond to this in several ways like

" I cant do that" or "That sounds hard" or a simple chuckle with a "Nooooo". Something that will elicit

"Awww, that’s cute" or "she can't do this" to maybe even "I should do something to help her do this" the situation most systems gravitate to.

Now, that to me is being on the way to developing a relationship with technology. Its not mere upsell and cross sell, it is building a relationship. This might seem like a sound argument but this option is not available to us.

Adult Voice

Alexa is an adult, so is Siri, Cortana and Google Now/Assistant.  We started with the robotic press # to continue and n-level deep menu, which put every single human being on the defensive with voice technology, to Siri a fully grown adult, which would constantly misunderstand utterings- leading to frustrations.  I struggled to get Siri to call my wife and went out of the way to add a digit to all my contacts whose names started with a V or a W. and there was one contact in the address book for "Wife". 

As the voice of an adult, our expectations of technology is very different from our actual experience. Even though the technology delivers a marvelous advance in science, we are not impressed- mismatched expectations.

The Time is NOW.

As the conversation over beer continued, we discussed how Samsung's latest 2016 generation TV got  the TV interface right.  It took us many generations, co-evolution, iterations and incremental clunky advances to get the user experience right. The same will be true for Voice. We have to start now, learn how conversations work and then evolve, adapt, improve and perfect it.

Join me on this journey of taking technology to everyone- the natural way.

Feedback and your experience with VUI is really appreciated.

#ConversationDesign

Voice the new frontier

Amazon Echo has changed my worldview of technology. 

I do not make that statement lightly, having started my career coding GUI in C. It has been so long that the memory of drawing shapes on the screens, capturing mouse-down and mouse-up events was fun and exhilarating.  I was around to see the mouse work, GUI become really easy and program for touch with the advent of the iPhone.

The days of voice are here. Not a mere curiosity or geek-out fad; Voice Technology fundamentally changes the way technology interfaces with us humans. Yes, you read that right- the tables have turned and it is up to the machines to work with us in our ways of doing things.

The Approaches to voice technology.

Just over a month of research, has yielded two clear approaches to voice technology. 

1. Artificial Intelligence with Natural Language Processing (NLP)
2. Artificial Intelligence with Machine Perception (MP)

It is unfortunate that voice technology is clubbed into "Artificial Intelligence" in the sense that the advances seem lackluster in view of the grand objectives of AI. The strides have been great and deserve some more explanation. NLP according to wikipedia gives machines the ability to  read and "understand" the languages humans speak. This definition alludes to the fact that machines will have sufficient power and processing to just pick-up whatever we humans have created and use it right off the bat. I believe there is a lot of work to be done to realize this vision or might be a pipe dream.

MP working in the domain of speech recognition is the more promising technology and has caught my attention this past month. In the MP world, machines- imperfect as they might be are able to reasonably translate human utterances into text and act on the interpretation- provided we guide the machine appropriately. Amazon Echo is a clear demonstration of this technology at work.

E-marketer estimates about 2.9 million Amazon Echo devices being shipped. Throw-in the 100's of millions of iPhones with Siri and Android capable of Google Assistant and we have a voice market ripe for picking.

The Catch

Amazon Echo has demonstrated that with proper programming, you can get a lot done with the Voice User Interface. Programming is required; it is absolutely essential to design applications that cater to the Voice Interface. The machines adapting themselves to work with voice is not going to happen anytime soon. When the mouse came on the scene, we had to program the input to work with the click and double-click. The older generation Text did not do the job by itself- the same holds true for voice as an interface.

The Opportunity

I see a huge opportunity to convert applications over to voice. Applications that will work with various systems from several vendors deploying various strategies to appropriate the market for themselves. There will also be a open-source effort sometime very soon. Millions of people around the world will work to teach the machines to work with Voice and as the technology matures add more capabilities and simplify our lives. 

Get a Head Start

To benefit from this opportunity, i would invite you to get started. We will not be building an improved IVR or a more natural sounding robot. We have to start by learning the mechanics of conversation- an awareness of how we communicate, interact and work. This will enable us to build interactions the machines can follow to naturally satisfy users without using a screen, keyboard, touch, click or any other unnatural mechanism we invented to work around the technology limitations of the past century.

Join me on this journey of taking technology to everyone- the natural way.

Feedback and your experience with VUI is really appreciated.

#ConversationDesign