# WhisperD: Dementia Speech Recognition and Filler Word Detection with Whisper

arXiv: https://arxiv.org/abs/2505.21551 · HTML: https://mlsanity.com/paper/2505.21551/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 33 references that are in the map.
- First posted on arXiv 25 May 2025.

## Builds on (most cited first, 6 of 6)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [CrisperWhisper: Accurate Timestamps on Verbatim Speech Transcriptions](https://mlsanity.com/paper/2408.16589/) (2024)
- [Alzheimer's Dementia Recognition Using Acoustic, Lexical, Disfluency and Speech Pause Features Robust to Noisy Inputs](https://mlsanity.com/paper/2106.15684/) (2021)
- [Conformer Based Elderly Speech Recognition System for Alzheimer's Disease Detection](https://mlsanity.com/paper/2206.13232/) (2022)
- [Filler Word Detection and Classification: A Dataset and Benchmark](https://mlsanity.com/paper/2203.15135/) (2022)
- [Prosody-Driven Privacy-Preserving Dementia Detection](https://mlsanity.com/paper/2407.03470/) (2024)

## Built on by (most cited first, 2 of 2)

- [Learning to Hear Hesitation: Continual Learning for Disfluency-Aware ASR](https://mlsanity.com/paper/2606.14391/) (2026)
- [Beyond WER: Entity and Disfluency Recall in Accented Conversational ASR](https://mlsanity.com/paper/2609.20828/) (2026)

## Abstract (arXiv)

Whisper fails to correctly transcribe dementia speech because persons with dementia (PwDs) often exhibit irregular speech patterns and disfluencies such as pauses, repetitions, and fragmented sentences. It was trained on standard speech and may have had little or no exposure to dementia-affected speech. However, correct transcription is vital for dementia speech for cost-effective diagnosis and the development of assistive technology. In this work, we fine-tune Whisper with the open-source dementia speech dataset (DementiaBank) and our in-house dataset to improve its word error rate (WER). The fine-tuning also includes filler words to ascertain the filler inclusion rate (FIR) and F1 score. The fine-tuned models significantly outperformed the off-the-shelf models. The medium-sized model achieved a WER of 0.24, outperforming previous work. Similarly, there was a notable generalisability to unseen data and speech patterns.