BluArmor Help - The Integrated Voice Assistant on Blu3 E20 that works with Siri/Google Assistant

BluArmor’s vision is to offer car-like comfort to two-wheeler riders. When we were evaluating different methods for the rider to interface with our next flagship product, BLU3 E20, we felt the right place for us to start would be to look at how cars approach the problem today. A significant development in car-infotainment systems in recent years has been the focus on voice as the primary interface. Voicebot.ai’s 2019 report (key findings specific to in-car voice assistant usage shown in Figure 1) reveals that 2 out of 3 consumers already use them in their cars at least monthly – a frequency we believe will increase significantly as the technology continues to become more powerful, predictable and user-friendly. 

 

In keeping with these developments, we made voice the primary user interface for BLU3 E20. We then had two possible ways forward in front of us – we could either leverage the ever-growing capabilities of the phone’s voice assistant by integrating our function-set with Siri/Google Assistant or we could build our own voice assistant. Figure 2 shows a simple pictorial representation of a voice assistant. 

How Does a Typical Voice Assistant Work?

 

To effect a user request, the voice assistant must first convert the user’s speech to text (automated speech recognition), analyze the text while factoring in the context (intent recognition), execute the request, and finally convert its response from text to speech and play it back to the user. Voice assistants continuously improve on their ability to execute the user requests through a “training” process that is compute-and-storage intensive, not to mention requires access to millions of voice samples. Industry leaders like Apple, Amazon, and Google are investing billions of dollars every year to improve the accuracy and effectiveness of their respective voice assistants, and the progress they are making year-on-year is astounding. This made our decision very simple – it was a no-brainer for us to conclude that we would integrate our function-set with Siri/Google Assistant; any and all improvements in Siri/Google Assistant’s effectiveness would directly translate to a better BLU3 E20 “experience” for our riders.

Key Features of BluArmor Voice Assistant

We realized that we wouldn’t be able to exploit the full power of the voice assistant unless it was easy to access, and built a click-surface powered by our proprietary SAeFEx technology that gave riders one-click access to Siri/Google Assistant. We then turned our attention to the different actions that Siri/Google Assistant could help riders with when they were on the road and identified four categories.

1. Control/Program BLU3 E20: Turn on/off the fan, regulate the fan speed, query the battery percentage, call your pillion and much more.
2. Access BLU3 E20’s Manual: a list of “how-do-I” questions that you can ask the assistant such as “how do I make a pillion call” or “how do I respond to a Whatsapp message”. Siri/Google Assistant will respond with detailed instructions.
3. Access Apps on the phone: make calls, send Whatsapp messages, play music, navigate to a destination, or search for nearby restaurants or gas stations.
4. Access predefined routines/shortcuts: access “Commute to Work routine” or “Commuting Back Home routine” which gives you quick access to the things that you would like to do on your frequent rides to Work or Home. 

Achieving fan control through the Voice Assistant

Let us look at a specific example in more detail - say, the rider wants to turn on BLU3 E20’s fan using the Google Assistant. For this request to be executed correctly, the following steps are necessary

• Google Assistant must convert the speech to text and identify the intent correctly
• It must then associate the intent with BLU3 E20 and pass it on to BLU3 E20’s companion app
• BLU3 E20’s companion app should instruct the firmware on BLU3 E20 to execute the command (in this case, turning on the fan)
• The firmware turns on the fan and notifies the app that the command was successfully executed
• The app then converts this to speech and tells the rider that the request was successfully executed

 

Figure 3: Sequence diagram for achieving device actions like fan control using the Google Assistant

Even a simple request such as “turn-on-the-fan” requires a fair bit of analysis and a number of instruction-handoffs to work seamlessly. We have achieved this by building a Voice Actions Framework (Figure 3) as an extension to the Google Assistant. This enables us to “connect” the different commands that the rider asks the Google Assistant to execute in the companion app (in other words, ensure intent exchange between the Google Assistant and the companion app). The app then instructs the firmware (and receives confirmation) using Bluetooth Low Energy (BLE) services built on the connect technology that powers BLU3 E20.

Supporting "How-do-I" questions for riders using the Voice Assistant

One of BLU3 E20’s unique features is that it lets riders access the user manual on-the-go; riders can ask Siri/Google Assistant a number of “how-do-I” questions on BLU3 E20 and receive detailed instructions. It is interesting to note here that if one were to ask Siri “how do I make a phone call?”, it responds with the results of a web search. In our opinion, this is not very effective when someone is looking for a specific answer to his/her question, and we have addressed this gap using the Siri Shortcut feature and marrying it effectively with Dialogflow. Dialogflow is Google’s Intent Recognition engine and uses complex AI models for Natural Language Understanding (NLU) making it easy for apps to provide conversational experience across all platforms.

To understand our approach better, let us look at an example, say the rider wants to know how he/she calls the pillion. This is a two-step process for the rider

Step 1: Invoke Siri, and say “BluArmor Help me”. This sets the context for Siri through a shortcut. Siri is now aware that the rider is looking to access BLU3 E20’s manual.

Step 2: Ask the “how-do-I” question - in this case, “How do I call my pillion?”

 

 

The following steps now get implemented 

• Siri translates the speech to text and sends the text query to Siri Shortcut
• Siri Shortcut sends an HTTP request to Dialogflow API to get the response to the query 
• Siri Shortcut sends the message received from Dialogflow API back to Siri
• Siri translates the text into a voice response, which completes the request

With riding comfort in mind, we have extended the range of “BluArmor Help me” to include basic help on the assistant. For example, riders can now use the “BluArmor Help me” feature to ask questions like “How do I make a phone call?” and will hear specific instructions from Siri/Google Assistant in response as opposed to hearing the results of a web search.

BluArmor Help - The Road Ahead

With BLU3 E20, we believe we are just starting to unravel the world of possibilities that exist when a connected-product leverages the potential of the voice assistant. The time is not far when a connected riding-accessory like BLU3 E20 initiates several rider-friendly routines that are closely integrated with the voice assistant based on the rider’s situation (start of a ride, being stuck in traffic, being in an accident, etc.). Siri/Google Assistant supports some of these routines today (as long as the user is willing to invest the time to set up a routine), and so does BLU3 E20 through its companion app.

The rider universe can expect to see a host of new, rider-aware features being unlocked as these two systems learn to engage with each other better and better.

Excited? Want to get one right away? Hang on a bit as we get BLU3 E20 ready for production. If you are interested in knowing more, you can sign up here. We will keep you posted on our production-readiness and share exciting early-bird offers (opportunity to beta-test the product, steep early-bird discounts, etc.)