# Benchmarking Commercial Speech Recognition and Multimodal Large Language Models on Dysarthric Speech: Severity-Stratified Baselines and Architecture-Specific Prompting Effects

arXiv: https://arxiv.org/abs/2512.17474 · HTML: https://mlsanity.com/paper/2512.17474/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Dec 2025.

## Built on by (most cited first, 3 of 3)

- [When Audio-Language Models Fail to Leverage Multimodal Context for Dysarthric Speech Recognition](https://mlsanity.com/paper/2605.02782/) (2026)
- [Comparing Human and Automatic Recognition of Dutch Dysarthric Continuous Speech: A Case Study](https://mlsanity.com/paper/2606.30237/) (2026)
- [Benchmarking Human and Automatic Speech Recognition of Diverse Speech: Initial Results](https://mlsanity.com/paper/2607.19049/) (2026)

## Abstract (arXiv)

Voice-based human-machine interaction has become a primary means of accessing intelligent systems, yet individuals with dysarthria are systematically excluded by persistent gaps in recognition accuracy. Although automatic speech recognition (ASR) achieves word error rates (WER) below 5% on typical speech, performance degrades sharply for dysarthric speakers, while the zero-shot behaviour of multimodal large language models (MLLMs) on such speech remains unclear. We evaluate eight commercial speech-to-text services on the TORGO dysarthric speech corpus: four conventional ASR systems (AssemblyAI, Whisper large-v3, Deepgram Nova-3, Nova-3 Medical) and four MLLM-based systems (GPT-4o, GPT-4o Mini, Gemini 2.5 Pro, Gemini 2.5 Flash), using lexical accuracy, semantic preservation, and cost-latency measures. Recognition degraded consistently with severity. Mild dysarthria reached low single-digit WER, around 1-2% for the leading systems, whereas severe dysarthria exceeded 51% WER for every system, with no MLLM advantage over conventional ASR under default settings. A four-condition prompt ablation showed architecture-specific effects: for the OpenAI models, verbatim-transcription prompts reduced severe-tier WER mainly by suppressing non-target-language drift, lowering GPT-4o from 60.1% to 52.9% and GPT-4o Mini from 66.0% to about 55%; Gemini models showed no consistent benefit and sometimes degraded. Semantic metrics correlated strongly with WER and were largely redundant in aggregate, but identified cases where communicative intent was partly preserved despite poor lexical accuracy. These severity-stratified, per-speaker baselines provide a reusable reference for evidence-based technology selection in assistive voice interfaces.