• Research

Guidelines for Designing an In-Car Voice Assistant

September, 2022

12 min read

bannerthumbnail

I spent the last couple of months researching the role of voice assistants in reducing driver distraction. I discovered that voice assistants have the potential to reduce distraction if they are designed well. Having covered that theoretical background in my previous post, I collected all the research and turned it into concrete guidelines for designing a voice assistant for drivers.

General design

Before designing the interaction sequence with the voice assistant, consider its context inside the car. Voice interaction will impact the rest of the in-car system and the behavior of the driver. When implementing the voice assistant, there are a couple of guidelines to consider.

1. Auditory information should come from the same location as visual information

People find it difficult to divide attention spatially. Drivers are focused on scanning the road. To optimize the multitasking capability of the driver, the audio from the assistant should come from the same location as that visual information, which is in front of the driver.

2. Always show the state of the system

Following Nielsen's first usability heuristic, drivers should be aware of the state of the system to avoid confusion. The system should communicate when it is receiving, processing, and outputting information. Furthermore, it should clearly show error states. For example, while driving, an internet connection is often unstable. It should be clear to the driver that the sequence takes longer because of that to prevent errors.

3. Allow for voice and manual activation

A driver should be able to trigger and interrupt an interaction via voice and manual input. The voice activation can be a specific phrase, like 'Hey Mercedes', or a direct command, like 'open Spotify'. Especially in the case of voice activation, the system must respond immediately to the activation.

4. Be aware of visual distraction

Interacting with a voice assistant can lead to visual distraction for multiple reasons.

  1. When a task requires a lot of cognitive resources, it can impact the driver's multitasking capabilities leading to a reduction in visual processing.
  2. Some drivers tend to direct their gaze towards the 'source' of the voice assistant when speaking (for example the center display)
  3. Drivers may check the visual output of the prompt on the display to confirm if an action is performed correctly.

Make sure that an interaction sequence with the voice assistant does not cause unnecessary visual distraction.

NIO uses a visual representation of the voice assistant that may cause more visual distraction
NIO uses a visual representation of the voice assistant that may cause more visual distraction

Tightly integrate the voice assistant with the rest of the interface

Depending on the implementation, a voice assistant can be an essential part of the in-car experience, or an unused gimmick (as most systems are today). Even though technically, a voice assistant is often separate from the rest of the infotainment system, it should be well integrated from the viewpoint of the driver.

5. Choose the best input and output modality

Not every use case is best via voice command. Test what is the best modality for each interaction. An interaction sequence likely relies on multiple modalities.

For example, saying the name of a location is easier than typing it. On the other hand, choosing a route may be easier on a display than via voice.

Do

Don't

DRIVER

Navigate to Berlin

DRIVER

Navigate to Berlin

ASSISTANT

OK, tap your preferred route on the display

ASSISTANT

OK, there is a route via highway A9 or A93, which would you like?

6. Allow drivers to reference any element that is visible on the display

Drivers will often refer to items visible on the display. This can be by name or common term for the item. For example, a driver may ask "how long is the disruption" when looking at an upcoming traffic jam on the navigation system. The assistant should be able to infer what the driver is talking about.

7. Don't make the voice UI full screen

Timely information like directions or warnings should not be obstructed. The voice assistant should show its state, but without covering other elements. The driver should be able to reference everything on the screen while the assistant is active.

Apple changed the CarPlay Siri UI to prevent it from hiding other elements
Apple changed the CarPlay Siri UI to prevent it from hiding other elements

Make it personal

Humans aren't computers. They communicate with each other using ambiguous language, filler words, emotion, etc. The same is expected in communicating with a voice assistant. The more 'human' a voice assistant behaves, the higher the trust and satisfaction.

8. Vary responses

Monotonous dialogs increase sleepiness. In repetitive tasks, the responses of the assistant should be varied. Having a varied, engaging assistant can actually reduce the effects of sleepiness. Additionally, the assistant should understand and use 'human' conversation patterns like ambiguity and filler words. Also, always use the active tense and find the right formality, which may be dependent on culture.

Do

Don't

DRIVER

When is my first meeting?

DRIVER

When is my first meeting?

ASSISTANT

You have a call with Alice at 9 am

ASSISTANT

Your first meeting starts at 9 am

DRIVER

When was my first meeting again?

DRIVER

When was my first meeting again?

ASSISTANT

Your first meeting is a call with Alice at 9 am

ASSISTANT

Your first meeting starts at 9 am

9. Match the personality and emotions of the driver

Much research has shown that matching the personality and the emotional state of the driver can greatly increase trust and satisfaction. Be careful with this though, a mismatch between personalities and emotions can have an adverse effect.

10. Be aware of the context of the driver

The interaction can be greatly improved if the assistant is aware of the driver's context, such as location, weather, number of passengers, and previous interactions.

Do

Don't

DRIVER

Navigate to the supermarket

DRIVER

Navigate to the supermarket

ASSISTANT

Last time, you parked at a nearby charger. Do you want to do this again?

ASSISTANT

I added the supermarket as a destination

DRIVER

Oh no, let's stop at the charger nearby

ASSISTANT

Which charger do you prefer?

Error handling

One of the main challenges of voice assistants is dealing with errors. Compared to a visual interface, a communication breakdown is harder to correct and results in more stress. That is why there is a greater focus on preventing errors. For each interaction sequence, the designer should keep in mind and test what possible errors may occur and find ways to prevent them.

11. Ask direct questions

Make it easy for the driver to reply to a question. Avoid using open-ended questions to prevent confusion or unwanted answers.

Do

Don't

DRIVER

Navigate to a burger place nearby

DRIVER

Navigate to a burger place nearby

ASSISTANT

There is a Burger King and McDonald's, which one would you like?

ASSISTANT

Would you like Mcdonald's or Burger King?

DRIVER

Burger King

DRIVER

Yes

12. Prevent error loops, always vary the prompt if misunderstood.

When drivers don't understand what went wrong, they may repeat the same answer, causing the error again. Always vary the prompt after a misunderstanding to provide more context and clues for what answers are appropriate.

Do

Don't

DRIVER

Navigate to my Hotel

DRIVER

Navigate to my Hotel

ASSISTANT

I did not catch that, what is the name of the place you want to go to?

ASSISTANT

I did not catch that, where do you want to go?

DRIVER

Grand Plaza Hotel

DRIVER

Navigate to my Hotel

However, when re-prompting, always give the same context again. When a dangerous traffic situation occurs, drivers cognitively 'shut off' listening to the voice assistant. Assume that the driver did not hear anything.

Do

Don't

ASSISTANT

It looks like there is traffic on the way, I found a 10-minute faster route. Would you like to change to this route?

ASSISTANT

It looks like there is traffic on the way, I found a 10-minute faster route. Would you like to change to this route?

DRIVER

What?

DRIVER

What?

ASSISTANT

There is a traffic jam, a 10-minute faster route is available. Would you like to change to this route?

ASSISTANT

Would you like to change to the faster route?

13. Adapt to the level of the user

Beginners will need more information to be able to understand the limitations of the system. Assist these users by communicating what is possible and by offering ways to improve the interaction. If a driver uses an unknown command, offer alternatives instead.

Do

Don't

DRIVER

Order a pizza

DRIVER

Order a pizza

ASSISTANT

I'm only able to help you navigate to a pizza restaurant, do you want to do that?

ASSISTANT

I can't do that

Adapt to the driving context

As explained in my previous post, reducing distraction should be the main goal of the designer. Voice assistants are a promising technology in this regard, if designed well. There are a couple of things to take into account when designing a voice assistant for driving that are different from typical voice assistant scenarios.

14. Keep the number of interactions to a minimum.

The driver's main focus is to drive safely. The voice assistant should communicate as efficiently as possible to minimize the time the driver spends interacting with it. Here are three examples of how to achieve that:

  1. Keep interactions brief and in natural language

Do

Don't

DRIVER

Open Audible

DRIVER

Open Audible

ASSISTANT

Which book do you want to listen to?

ASSISTANT

Let's listen to a book. Say the name of the book you want to listen to or say 'find new book' to let me help you find new books to listen to

  1. Do not ask unnecessary questions.

Do

Don't

DRIVER

Find the nearest charger

DRIVER

Find the nearest petrol station

ASSISTANT

OK, it will take two minutes to get there

ASSISTANT

OK, do you want to add this as a favorite?

DRIVER

No

ASSISTANT

OK, I added it as a destination, it will take two minutes to get there

  1. Don't add prompts when they are not necessary.

Do

Don't

DRIVER

Play my driving playlist

DRIVER

Play my driving playlist

ASSISTANT

*plays playlist*

ASSISTANT

Do you want to play the playlist called 'driving'?

DRIVER

Yes!

ASSISTANT

OK, playing 'driving'

15. Deliver a message when the driver is not engaged

A voice message is delivered ideally when the car is in a calm environment. Audio messages are perceived to be more important than visual ones and capture the attention more. Delivering a message at a critical moment may cause the driver to fail to react appropriately or ignore the message.

16. Wait longer because the driver may be otherwise engaged

If an interaction happens during a critical moment, the driver will wait to respond until the situation improves. Be mindful that producing speech requires more cognitive resources than listening. The driver can pause mid-sentence or wait to reply for longer than usual. The voice assistant should take this into account before canceling the sequence altogether.

17. Let drivers control the voice assistant

Consider giving drivers control over the pace of the interaction. Allow them to easily pause or interrupt the sequence so they can continue when they prefer.


And last...

18. Play around with voice assistants!

If you are a designer in the car industry, you should take voice assistants seriously and include them early on in your design process. It is a lot easier than you may think. There are several great tools available to prototype and play around with voice interaction. I like using ProtoPie, it is specifically aimed at automotive applications. There are tons of opportunities to improve the in-car interaction with voice assistants but you will only know what's possible until you have played around with it!

Made in Berlin

This website is made from scratch with React, Gatsby, and Tachyons. It is typeset in Inter. This website stores zero cookies and uses Simple Analytics to guarantee your privacy