# Aphasic Speech Recognition using a Mixture of Speech Intelligibility Experts

arXiv: https://arxiv.org/abs/2008.10788 · HTML: https://mlsanity.com/paper/2008.10788/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal MoE models, one of 12 directions in Mixture-of-experts.
- First posted on arXiv 25 Aug 2020.

## Placed in

- Mixture-of-experts / [Multimodal MoE models](https://mlsanity.com/moe/multimodal-moe-models/)

## Built on by (most cited first, 4 of 4)

- [A New Benchmark of Aphasia Speech Recognition and Detection Based on E-Branchformer and Multi-task Learning](https://mlsanity.com/paper/2305.13331/) (2023)
- [Seq2seq for Automatic Paraphasia Detection in Aphasic Speech](https://mlsanity.com/paper/2312.10518/) (2023)
- [On-the-fly Routing for Zero-shot MoE Speaker Adaptation of Speech Foundation Models for Dysarthric Speech Recognition](https://mlsanity.com/paper/2505.22072/) (2025)
- [AS-ASR: A Lightweight Framework for Aphasia-Specific Automatic Speech Recognition](https://mlsanity.com/paper/2506.06566/) (2025)

## Abstract (arXiv)

Robust speech recognition is a key prerequisite for semantic feature extraction in automatic aphasic speech analysis. However, standard one-size-fits-all automatic speech recognition models perform poorly when applied to aphasic speech. One reason for this is the wide range of speech intelligibility due to different levels of severity (i.e., higher severity lends itself to less intelligible speech). To address this, we propose a novel acoustic model based on a mixture of experts (MoE), which handles the varying intelligibility stages present in aphasic speech by explicitly defining severity-based experts. At test time, the contribution of each expert is decided by estimating speech intelligibility with a speech intelligibility detector (SID). We show that our proposed approach significantly reduces phone error rates across all severity stages in aphasic speech compared to a baseline approach that does not incorporate severity information into the modeling process.