# Alzheimer's Dementia Recognition Using Acoustic, Lexical, Disfluency and Speech Pause Features Robust to Noisy Inputs

arXiv: https://arxiv.org/abs/2106.15684 · HTML: https://mlsanity.com/paper/2106.15684/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 25 references that are in the map.
- First posted on arXiv 29 Jun 2021.

## Builds on (most cited first, 5 of 5)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Highway Networks](https://mlsanity.com/paper/1505.00387/) (2015)
- [Detecting cognitive decline using speech only: The ADReSSo Challenge](https://mlsanity.com/paper/2104.09356/) (2021)
- [Multi-modal fusion with gating using audio, lexical and disfluency features for Alzheimer's Dementia recognition from spontaneous speech](https://mlsanity.com/paper/2106.09668/) (2021)
- [Re-framing Incremental Deep Language Models for Dialogue Processing with Multi-task Learning](https://mlsanity.com/paper/2011.06754/) (2020)

## Built on by (most cited first, 27 of 27)

- [Detecting Dementia from Speech and Transcripts using Transformers](https://mlsanity.com/paper/2110.14769/) (2021)
- [Exploring linguistic feature and model combination for speech recognition based automatic AD detection](https://mlsanity.com/paper/2206.13758/) (2022)
- [Self-supervised ASR Models and Features For Dysarthric and Elderly Speech Recognition](https://mlsanity.com/paper/2407.13782/) (2024)
- [CogniAlign: Word-Level Multimodal Speech Alignment with Gated Cross-Attention for Alzheimer's Detection](https://mlsanity.com/paper/2506.01890/) (2025)
- [Exploiting prompt learning with pre-trained language models for Alzheimer's Disease detection](https://mlsanity.com/paper/2210.16539/) (2022)
- [Fillers in Spoken Language Understanding: Computational and Psycholinguistic Perspectives](https://mlsanity.com/paper/2301.10761/) (2023)
- [Context-aware attention layers coupled with optimal transport domain adaptation and multimodal fusion methods for recognizing dementia from spontaneous speech](https://mlsanity.com/paper/2305.16406/) (2023)
- [A Review of Deep Learning Approaches for Non-Invasive Cognitive Impairment Detection](https://mlsanity.com/paper/2410.19898/) (2024)
- [Transferring speech-generic and depression-specific knowledge for Alzheimer's disease detection](https://mlsanity.com/paper/2310.04358/) (2023)
- [A Digital Language Coherence Marker for Monitoring Dementia](https://mlsanity.com/paper/2310.09623/) (2023)
- [Reformulating NLP tasks to Capture Longitudinal Manifestation of Language Disorders in People with Dementia](https://mlsanity.com/paper/2310.09897/) (2023)
- [HAFFormer: A Hierarchical Attention-Free Framework for Alzheimer's Disease Detection From Spontaneous Speech](https://mlsanity.com/paper/2405.03952/) (2024)
- [A Systematic Review of NLP for Dementia -- Tasks, Datasets and Opportunities](https://mlsanity.com/paper/2409.19737/) (2024)
- [Performance Assessment of ChatGPT vs Bard in Detecting Alzheimer's Dementia](https://mlsanity.com/paper/2402.01751/) (2024)
- [Learning Co-Speech Gesture for Multimodal Aphasia Type Detection](https://mlsanity.com/paper/2310.11710/) (2023)
- [Speech Technology Services for Oral History Research](https://mlsanity.com/paper/2405.02333/) (2024)
- [Automatic Prediction of Amyotrophic Lateral Sclerosis Progression using Longitudinal Speech Transformer](https://mlsanity.com/paper/2406.18625/) (2024)
- [WhisperD: Dementia Speech Recognition and Filler Word Detection with Whisper](https://mlsanity.com/paper/2505.21551/) (2025)
- [Impact of Environmental Noise on Alzheimer's Disease Detection from Speech: Should You Let a Baby Cry?](https://mlsanity.com/paper/2203.17110/) (2022)
- [Deep Insights into Cognitive Decline: A Survey of Leveraging Non-Intrusive Modalities with Deep Learning Techniques](https://mlsanity.com/paper/2410.18972/) (2024)
- [CogBench: A Large Language Model Benchmark for Multilingual Speech-Based Cognitive Impairment Assessment](https://mlsanity.com/paper/2508.03360/) (2025)
- [Recognizing Dementia from Neuropsychological Tests with State Space Models](https://mlsanity.com/paper/2507.10311/) (2025)
- [Foundation Model-based Evaluation of Neuropsychiatric Disorders: A Lifespan-Inclusive, Multi-Modal, and Multi-Lingual Study](https://mlsanity.com/paper/2512.20948/) (2025)
- [What Do LLMs Know About Alzheimer's Disease? Multi-loss Fine-Tuning and Probing for AD Detection](https://mlsanity.com/paper/2602.11177/) (2026)
- [Behind the [MASK]: Disentangling Representation and Faithfulness in DAPF-Based Dementia Detection](https://mlsanity.com/paper/2608.25028/) (2026)
- [Why Alzheimer's Speech Screening Fails to Generalize: Bridging the Deployment Gap via Cross-Corpus Evidence Anchoring](https://mlsanity.com/paper/2609.31293/) (2026)
- [Explainable and Generalisable LLM-based Cognitive Decline Detection with Spontaneous Speech](https://mlsanity.com/paper/2609.34217/) (2026)

## Abstract (arXiv)

We present two multimodal fusion-based deep learning models that consume ASR transcribed speech and acoustic data simultaneously to classify whether a speaker in a structured diagnostic task has Alzheimer's Disease and to what degree, evaluating the ADReSSo challenge 2021 data. Our best model, a BiLSTM with highway layers using words, word probabilities, disfluency features, pause information, and a variety of acoustic features, achieves an accuracy of 84% and RSME error prediction of 4.26 on MMSE cognitive scores. While predicting cognitive decline is more challenging, our models show improvement using the multimodal approach and word probabilities, disfluency and pause information over word-only models. We show considerable gains for AD classification using multimodal fusion and gating, which can effectively deal with noisy inputs from acoustic features and ASR hypotheses.