“Alexa, who is the Prime Minister of Canada?” asked Justin, one of members of the Amazon Alexa team on a Tuesday afternoon. The team made a stop at the University of Massachusetts Boston on Sept. 11 and 12 to introduce us to Cleo, a conversational tool and game to help Alexa learn languages and cultures from around the world by just having a conversation with her.
Before we got to Cleo, however, the team gave an update on what Alexa is capable of as of now. In response to the above question, Alexa replied, “Canada’s Prime Minister is Justin Trudeau.” Alexa then carried on a conversation about Prime Minister Trudeau without Justin having mentioned his name at all. This was to demonstrate that Alexa is now capable of recognizing context and will understand who or what we’re talking about when we want to get information from Alexa. She can also remember things such as someone’s favorite color and recite it back to you six months, one year, or even five years later. She can also give you the name of a song even if a single line or phrase is the only thing you can remember. Alexa also has the Alexa Skills Kit, which are apps, or skills, for Alexa developed by third-party developers for added functionalities. One example was to ask Alexa to order you a coffee from your local Starbucks, and it would be ready for you to pick up from the counter without having to stand in line, take out your money, or even talk to anybody, which is great if you’re a Grumpy Gus without your daily morning hit of caffeine.
There are more than 35,000 such Alexa skills for you to choose from or you can even create one of your own. One of the skills of Alexa is Cleo, where we can help Alexa learn more about our language and culture so that Alexa can meet the whole world instead of just a few countries.
The talk then moved on to how Alexa processes what we ask her and gives us the answer. Here’s the rundown without going all nerdy, hopefully. Alexa uses a wake-word engine which “wakes” Alexa up when it hears the word “Alexa.” It then listens to what we say, trying to separate what we actually say from ambient noises such as background noise, differentiating acoustically similar words, and a host of other things. Then, Alexa turns the speech to text, picks out key words, finds the answer, returns the text back to speech and gives us the answer, all in just a matter of one to two seconds.
We had a member of the team, Gerard, come up to give a demonstration of Alexa “learning” Spanish. When Cleo was opened and once Alexa knew what language and from where she was going to learn, she then told us scenarios in the form of a story which, in our case, was a story about moving to a new apartment. Alexa asked the user to ask questions according to what she said, such as asking Alexa to check our to-do list, what the time was, turn on the lights, and so on. The replies were all given in Spanish in a conversational tone and speed. The game stopped after 10 rounds and Alexa asked if Gerard wanted to continue. Gerard declined, and the presentation continued.
After the short demonstration, there was some time for Q&A. One audience member wanted to know if someone else used our device which was connected to our financial accounts, would they be able to buy something, to which the answer was that Alexa has voice recognition and will be able to recognize whether the authorized speaker is the one who’s giving the commands. Another question was whether Alexa would be able to recognize commands given by bilingual or multilingual users who might switch languages while giving commands. The answer to that was that Alexa hasn’t learned to tackle language switching yet but hopefully with a lot more good data and training, she will learn to. I then asked what if users use swear words when they use Cleo to teach Alexa a new language. The team said that they hope to catch them and hopes Alexa learns to understand what they mean if a user does swear and doesn’t say it back on another occasion. After answering questions for a few more minutes, they ended the session by handing out a some Amazon merchandise and an opportunity to meet and greet the team.
Anyone can install Cleo and teach it your own language and even different accents; for example, telling Alexa you want to teach her English from New Zealand, England, India, Brazil, etc. The aim of the Alexa team is to learn as many languages and accents it can so that Alexa can go international and be incorporated into our everyday lives. One little-known, exciting thing is that you don’t even have to buy an Amazon device to use Alexa! You can install the Amazon Alexa app from the app store for iOS and Android devices. So, install, open, and teach Alexa your language and culture, but remember to mind your language.