Adding Voice Control to your Mobile App
February 11th, 2020 by Ishan GeeganageAllowing the user of your mobile app to instruct it via Google Voice Assistant or Siri is a great way of making it easier for your user to get on with their day, and get the answers or action they need to accomplish done quickly, or when their hands aren’t free or glasses nowhere in sight.
Access to Google Voice Assistant (Android) or Siri (Apple) are governed by the methods available to interact with these APIs, and that means voice control is constrained to very particular functions – although the available functions has been significantly extended since we last wrote on this topic back in 2016. Below is a list of the areas in which we can currently use voice activation / control:
Google Voice Assistant
| Domain | Supported Actions |
| Alarm | Set Alarm Set Timer |
| Communication | Initiate a phone call |
| Fitness | Start/Stop a bike ride Start/Stop a run Start/Stop a workout Show heart rate Show step count |
| Location | Book a Cab |
| Media | Play music from search Take a picture Record a video |
| Open | Open URL Open App |
| Productivity | Take a Note |
| Search | Search using a specific app |
Siri
| Domain | Supported Actions |
| VoIP calling | Initiate Call Search the call history |
| Messaging | Send Message |
| Payments | Send payments. Request payments. Pay bills. Search for bills. Search for and view account information, including balances, points, and miles. Transfer money between accounts. |
| Photos management | Search for photo and show them in the app |
| Fitness activities | Start, pause, resume, end and cancel workout |
| Ride Booking | Book rides Provide ride status information |
| CarPlay integration (not supported on watchOS) | Activate and save a drivers setting Change the cars audio source Change the cars climate Change the cars defroster settings. Change the cars seat settings. Change the cars radio station. |
| Vehicle integration | Activate hazard light or honk the horn Lock and unlock the doors. Check the current fuel or power level. |
| List and Notes | Create to-do lists and items. Search for to-do lists and items. Mark to-do list items as complete. Create reminders based on a date, time, and/or location. Create notes. Search for notes. Modify notes. |
| Media | Search for and play music, audiobooks, and podcasts. Like or dislike items. Add items to a library or playlist |
| Restaurant Reservation | Create and manage restaurant reservations |
| Visual Codes | Show a visual code, like a QR code or bar code |
In every main OS version release, we expect there will be new opportunities added into these services. This blog was written based on:
- Sirikit support iOS version 10.0
- Google Voice Action support version 21
It is also important to note that some actions can only be implemented in either Android or Apple, but not both e.g. search can only be carried out in your Android app not in your iOS app. In iOS you can scan a QR code with your voice, but in Android you cannot.
How is Voice Control added to my App?
In order to allow voice control of your app, the app must be programmed specifically to be accessible by voice control and particular Actions listed above. When a person downloads your app and installs it on their device, they must agree to voice control being allowed. Then when a voice command is received by your phone that relates to one of the domains & supported actions listed above, and which has a responding activity within your app, then the functionality of your app can be triggered.
What Can I Make My App Do After Receiving a Voice Command?
If your app wants to access any of the standard apps that can be controlled by voice, then your app can be programmed to interact with those standard apps upon receiving a voice command. For example, in your custom app, if you wanted to take a piece of content from that app and save it to a Note (in-built device app), that can be achieved using voice commands.
You can’t talk to your app about something entirely outside the voice controls mentioned above. For example, you can’t say “Hey Google, in >mobile app name< open my transaction history.”
In some cases, your app will ask for confirmation before taking an action e.g. making payments, confirming you would like to make a particular booking, and so on. This is to ensure that the voice interpretation has correctly understood your request.
What Happens if there are Multiple Options?
Just say your mobile app contains a search feature, and you want to allow the user to use a voice command such as “Hey Siri, search for xyz”. Simultaneously of course your phone will also have at least one other search application. When your phone receives a search command, it will then ask you which app would you like to use for your search: x or y? Once you have answered that question, your search will be performed via the selected app. Alternatively, you could ask “Hey Siri, search >mobile app name< for xyz”.
Can I add a Custom Voice Command?
You might be tempted to try and work around the limited number of voice commands currently available e.g. “oh, I will use the ‘Book a Cab’ voice command, and have that do something completely different instead”. Both Google and Apple check your app before publishing it in the app stores, and we expect that they will reject your app if it has been programmed in this manner. In addition, if your app is not in the category of ride sharing, your app will not be accepted if it contains voice controls in the Ride Sharing App domain.
If you are looking to add voice control to your app, let’s have a chat!
by 
