# Deaf and Hard of Hearing Access to Intelligent Personal Assistants: Comparison of Voice-Based Options with an LLM-Powered Touch Interface

arXiv: https://arxiv.org/abs/2601.15209 · HTML: https://mlsanity.com/paper/2601.15209/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 42 references that are in the map.
- First posted on arXiv 21 Jan 2026.

## Builds on (most cited first, 9 of 9)

- [The Microsoft 2017 Conversational Speech Recognition System](https://mlsanity.com/paper/1708.06073/) (2017)
- [Let Me Do It For You: Towards LLM Empowered Recommendation via Tool Learning](https://mlsanity.com/paper/2405.15114/) (2024)
- [Recent Progress in the CUHK Dysarthric Speech Recognition System](https://mlsanity.com/paper/2201.05845/) (2022)
- [Deaf, Hard of Hearing, and Hearing Perspectives on using Automatic Speech Recognition in Conversation](https://mlsanity.com/paper/1909.01176/) (2019)
- [Automatic Speech Recognition Services: Deaf and Hard-of-Hearing Usability](https://mlsanity.com/paper/1909.02853/) (2019)
- [Large Language User Interfaces: Voice Interactive User Interfaces powered by LLMs](https://mlsanity.com/paper/2402.07938/) (2024)
- [Artificial Intelligence Fairness in the Context of Accessibility Research on Intelligent Systems for People who are Deaf or Hard of Hearing](https://mlsanity.com/paper/1908.10414/) (2019)
- [Assessment of Sign Language-Based versus Touch-Based Input for Deaf Users Interacting with Intelligent Personal Assistants](https://mlsanity.com/paper/2404.14605/) (2024)
- [Sign Language-Based versus Touch-Based Input for Deaf Users with Interactive Personal Assistants in Simulated Kitchen Environments](https://mlsanity.com/paper/2404.14610/) (2024)

## Abstract (arXiv)

We investigate intelligent personal assistants (IPAs) accessibility for deaf and hard of hearing (DHH) people who can use their voice in everyday communication. The inability of IPAs to understand diverse accents including deaf speech renders them largely inaccessible to non-signing and speaking DHH individuals. Using an Echo Show, we compare the usability of natural language input via spoken English; with Alexa's automatic speech recognition and a Wizard-of-Oz setting with a trained facilitator re-speaking commands against that of a large language model (LLM)-assisted touch interface in a mixed-methods study. The touch method was navigated through an LLM-powered "task prompter," which integrated the user's history and smart environment to suggest contextually-appropriate commands. Quantitative results showed no significant differences across both spoken English conditions vs LLM-assisted touch. Qualitative results showed variability in opinions on the usability of each method. Ultimately, it will be necessary to have robust deaf-accented speech recognized natively by IPAs.