Adding Voice Control to your Mobile App

February 11th, 2020 by Ishan Geeganage

Allowing the user of your mobile app to instruct it via Google Voice Assistant or Siri is a great way of making it easier for your user to get on with their day, and get the answers or action they need to accomplish done quickly, or when their hands aren’t free or glasses nowhere in sight.

Access to Google Voice Assistant (Android) or Siri (Apple) are governed by the methods available to interact with these APIs, and that means voice control is constrained to very particular functions – although the available functions has been significantly extended since we last wrote on this topic back in 2016. Below is a list of the areas in which we can currently use voice activation / control:

Google Voice Assistant

Domain Supported Actions
Alarm Set Alarm
Set Timer
Communication Initiate a phone call
Fitness Start/Stop a bike ride
Start/Stop a run
Start/Stop a workout
Show heart rate
Show step count
Location Book a Cab
Media Play music from search
Take a picture
Record a video
Open Open URL
Open App
Productivity Take a Note
Search Search using a specific app

Siri

Domain Supported Actions
VoIP calling Initiate Call
Search the call history
Messaging Send Message
Payments Send payments.
Request payments.
Pay bills.
Search for bills.
Search for and view account information, including balances, points, and miles.
Transfer money between accounts.
Photos management Search for photo and show them in the app
Fitness activities Start, pause, resume, end and cancel workout
Ride Booking Book rides
Provide ride status information
CarPlay integration (not supported on watchOS) Activate and save a drivers setting
Change the cars audio source
Change the cars climate
Change the cars defroster settings.
Change the cars seat settings.
Change the cars radio station.
Vehicle integration Activate hazard light or honk the horn
Lock and unlock the doors.
Check the current fuel or power level.
List and Notes Create to-do lists and items.
Search for to-do lists and items.
Mark to-do list items as complete.
Create reminders based on a date, time, and/or location.
Create notes.
Search for notes.
Modify notes.
Media Search for and play music, audiobooks, and podcasts.
Like or dislike items.
Add items to a library or playlist
Restaurant Reservation Create and manage restaurant reservations
Visual Codes Show a visual code, like a QR code or bar code

In every main OS version release, we expect there will be new opportunities added into these services. This blog was written based on:

  • Sirikit support iOS version 10.0
  • Google Voice Action support version 21

It is also important to note that some actions can only be implemented in either Android or Apple, but not both e.g. search can only be carried out in your Android app not in your iOS app. In iOS you can scan a QR code with your voice, but in Android you cannot.

How is Voice Control added to my App?
In order to allow voice control of your app, the app must be programmed specifically to be accessible by voice control and particular Actions listed above. When a person downloads your app and installs it on their device, they must agree to voice control being allowed. Then when a voice command is received by your phone that relates to one of the domains & supported actions listed above, and which has a responding activity within your app, then the functionality of your app can be triggered.

What Can I Make My App Do After Receiving a Voice Command?
If your app wants to access any of the standard apps that can be controlled by voice, then your app can be programmed to interact with those standard apps upon receiving a voice command. For example, in your custom app, if you wanted to take a piece of content from that app and save it to a Note (in-built device app), that can be achieved using voice commands.

You can’t talk to your app about something entirely outside the voice controls mentioned above. For example, you can’t say “Hey Google, in >mobile app name< open my transaction history.”

In some cases, your app will ask for confirmation before taking an action e.g. making payments, confirming you would like to make a particular booking, and so on. This is to ensure that the voice interpretation has correctly understood your request.

What Happens if there are Multiple Options?
Just say your mobile app contains a search feature, and you want to allow the user to use a voice command such as “Hey Siri, search for xyz”. Simultaneously of course your phone will also have at least one other search application. When your phone receives a search command, it will then ask you which app would you like to use for your search: x or y? Once you have answered that question, your search will be performed via the selected app. Alternatively, you could ask “Hey Siri, search >mobile app name< for xyz”.

Can I add a Custom Voice Command?
You might be tempted to try and work around the limited number of voice commands currently available e.g. “oh, I will use the ‘Book a Cab’ voice command, and have that do something completely different instead”. Both Google and Apple check your app before publishing it in the app stores, and we expect that they will reject your app if it has been programmed in this manner. In addition, if your app is not in the category of ride sharing, your app will not be accepted if it contains voice controls in the Ride Sharing App domain.

If you are looking to add voice control to your app, let’s have a chat!

Facebooktwitterredditpinterestlinkedinmailby feather

Leave a Reply

Subscribe to our monthly

Contactpoint Email News

Our enews is sent out approximately monthly, and contains information on latest digital technologies, and how these can be used to help your organisation grow.

To subscribe, simply fill in your details below: