# The State of Speech in HCI: Trends, Themes and Challenges

arXiv: https://arxiv.org/abs/1810.06828 · HTML: https://mlsanity.com/paper/1810.06828/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Oct 2018.

## Built on by (most cited first, 21 of 21)

- [Evaluating Human-Language Model Interaction](https://mlsanity.com/paper/2212.09746/) (2022)
- [LLaMA-Omni2: LLM-based Real-time Spoken Chatbot with Autoregressive Streaming Speech Synthesis](https://mlsanity.com/paper/2505.02625/) (2025)
- [Differentially Private Speaker Anonymization](https://mlsanity.com/paper/2202.11823/) (2022)
- [Paralinguistics-Aware Speech-Empowered Large Language Models for Natural Conversation](https://mlsanity.com/paper/2402.05706/) (2024)
- [A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairness](https://mlsanity.com/paper/2212.09006/) (2022)
- [The timing bottleneck: Why timing and overlap are mission-critical for conversational user interfaces, speech recognition and dialogue systems](https://mlsanity.com/paper/2307.15493/) (2023)
- [Raon-Speech Technical Report](https://mlsanity.com/paper/2605.23912/) (2026)
- [MyMove: Facilitating Older Adults to Collect In-Situ Activity Labels on a Smartwatch with Speech](https://mlsanity.com/paper/2204.00145/) (2022)
- [Transcending the "Male Code": Implicit Masculine Biases in NLP Contexts](https://mlsanity.com/paper/2304.12810/) (2023)
- [Designing AI Personalities: Enhancing Human-Agent Interaction Through Thoughtful Persona Design](https://mlsanity.com/paper/2410.22744/) (2024)
- [Toward Metaphor-Fluid Conversation Design for Voice User Interfaces](https://mlsanity.com/paper/2502.11554/) (2025)
- [Cross-Corpora Spoken Language Identification with Domain Diversification and Generalization](https://mlsanity.com/paper/2302.05110/) (2023)
- [Can Voice Assistants Sound Cute? Towards a Model of Kawaii Vocalics](https://mlsanity.com/paper/2304.12809/) (2023)
- ["I'm" Lost in Translation: Pronoun Missteps in Crowdsourced Data Sets](https://mlsanity.com/paper/2304.13557/) (2023)
- [What Do We See in Them? Identifying Dimensions of Partner Models for Speech Interfaces Using a Psycholexical Approach](https://mlsanity.com/paper/2102.02094/) (2021)
- [Coimagining the Future of Voice Assistants with Cultural Sensitivity](https://mlsanity.com/paper/2403.17599/) (2024)
- [Qualitative Approaches to Voice UX](https://mlsanity.com/paper/2404.14736/) (2024)
- [Large Language Model-assisted Speech and Pointing Benefits Multiple 3D Object Selection in Virtual Reality](https://mlsanity.com/paper/2410.21091/) (2024)
- [ChatGPT and U(X): A Rapid Review on Measuring the User Experience](https://mlsanity.com/paper/2503.15808/) (2025)
- [Super Kawaii Vocalics: Amplifying the "Cute" Factor in Computer Voice](https://mlsanity.com/paper/2507.06235/) (2025)
- [On the Linguistic and Computational Requirements for Creating Face-to-Face Multimodal Human-Machine Interaction](https://mlsanity.com/paper/2211.13804/) (2022)

## Abstract (arXiv)

Speech interfaces are growing in popularity. Through a review of 68 research papers this work maps the trends, themes, findings and methods of empirical research on speech interfaces in HCI. We find that most studies are usability/theory-focused or explore wider system experiences, evaluating Wizard of Oz, prototypes, or developed systems by using self-report questionnaires to measure concepts like usability and user attitudes. A thematic analysis of the research found that speech HCI work focuses on nine key topics: system speech production, modality comparison, user speech production, assistive technology & accessibility, design insight, experiences with interactive voice response (IVR) systems, using speech technology for development, people's experiences with intelligent personal assistants (IPAs) and how user memory affects speech interface interaction. From these insights we identify gaps and challenges in speech research, notably the need to develop theories of speech interface interaction, grow critical mass in this domain, increase design work, and expand research from single to multiple user interaction contexts so as to reflect current use contexts. We also highlight the need to improve measure reliability, validity and consistency, in the wild deployment and reduce barriers to building fully functional speech interfaces for research.