# Different Types of Voice User Interface Failures May Cause Different Degrees of Frustration

arXiv: https://arxiv.org/abs/2002.03582 · HTML: https://mlsanity.com/paper/2002.03582/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Feb 2020.

## Built on by (most cited first, 4 of 4)

- ["Stupid robot, I want to speak to a human!" User Frustration Detection in Task-Oriented Dialog Systems](https://mlsanity.com/paper/2411.17437/) (2024)
- [A Multimodal GUI Architecture for Interfacing with LLM-Based Conversational Assistants](https://mlsanity.com/paper/2510.06223/) (2025)
- [Unheard in the Digital Age: Rethinking AI Bias and Speech Diversity](https://mlsanity.com/paper/2601.18641/) (2026)
- [Understanding the Use of a Large Language Model-Powered Guide to Make Virtual Reality Accessible for Blind and Low Vision People](https://mlsanity.com/paper/2603.09964/) (2026)

## Abstract (arXiv)

We report on an investigation into how different types of failures in a voice user interface (VUI) affects user frustration. To this end, we conducted a pilot user study (n=10) and a main user study (n=30), both with a simple voice-operated calendar application that we built using the Alexa Skills Kit. In our pilot study, we identified three major failure types as perceived by the users, namely, Reason Unknown, Speech Misrecognition, and Utterance Pattern Match Failure, along with more fine-grained failure types from the developer's viewpoint such as Intent Pattern Match Failure and Intent Misclassification. Then, in our main study, we set up three user tasks that were designed to each induce a specific failure type, and collected user frustration ratings for each task. Our main findings are: (a)Users may be relatively tolerant to user-perceived Speech Misrecognition, and not so to user-perceived Reason Unknown and Utterance Mattern Match Failures; (b)Regarding the relationship between developer-perceived and user-perceived failure types, 68.8% of developer-perceived Intent Misclassification instances caused user-perceived Reason Unkown failures. From (a) and (b), a practical design implication would be to try to prevent Intent Misclassification from happening by carefully crafting the utterance patterns for each intent.