# A Tale of Two Perplexities: Sensitivity of Neural Language Models to Lexical Retrieval Deficits in Dementia of the Alzheimer's Type

arXiv: https://arxiv.org/abs/2005.03593 · HTML: https://mlsanity.com/paper/2005.03593/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 May 2020.

## Built on by (most cited first, 9 of 9)

- [GPT-D: Inducing Dementia-related Linguistic Anomalies by Deliberate Degradation of Artificial Neural Language Models](https://mlsanity.com/paper/2203.13397/) (2022)
- [TRESTLE: Toolkit for Reproducible Execution of Speech, Text and Language Experiments](https://mlsanity.com/paper/2302.07322/) (2023)
- [Slaves to the Law of Large Numbers: An Asymptotic Equipartition Property for Perplexity in Generative Language Models](https://mlsanity.com/paper/2405.13798/) (2024)
- [A Systematic Review of NLP for Dementia -- Tasks, Datasets and Opportunities](https://mlsanity.com/paper/2409.19737/) (2024)
- [Too Big to Fail: Larger Language Models are Disproportionately Resilient to Induction of Dementia-Related Linguistic Anomalies](https://mlsanity.com/paper/2406.02830/) (2024)
- [Mitigating Confounding in Speech-Based Dementia Detection through Weight Masking](https://mlsanity.com/paper/2506.05610/) (2025)
- [Alzheimer's Dementia Detection Using Perplexity from Paired Large Language Models](https://mlsanity.com/paper/2506.09315/) (2025)
- [Semantic Coherence Markers for the Early Diagnosis of the Alzheimer Disease](https://mlsanity.com/paper/2302.01025/) (2023)
- [Bigger But Not Better: Small Neural Language Models Outperform Large Language Models in Detection of Thought Disorder](https://mlsanity.com/paper/2503.20103/) (2025)

## Abstract (arXiv)

In recent years there has been a burgeoning interest in the use of computational methods to distinguish between elicited speech samples produced by patients with dementia, and those from healthy controls. The difference between perplexity estimates from two neural language models (LMs) - one trained on transcripts of speech produced by healthy participants and the other trained on transcripts from patients with dementia - as a single feature for diagnostic classification of unseen transcripts has been shown to produce state-of-the-art performance. However, little is known about why this approach is effective, and on account of the lack of case/control matching in the most widely-used evaluation set of transcripts (DementiaBank), it is unclear if these approaches are truly diagnostic, or are sensitive to other variables. In this paper, we interrogate neural LMs trained on participants with and without dementia using synthetic narratives previously developed to simulate progressive semantic dementia by manipulating lexical frequency. We find that perplexity of neural LMs is strongly and differentially associated with lexical frequency, and that a mixture model resulting from interpolating control and dementia LMs improves upon the current state-of-the-art for models trained on transcript text exclusively.