# Reformulating NLP tasks to Capture Longitudinal Manifestation of Language Disorders in People with Dementia

arXiv: https://arxiv.org/abs/2310.09897 · HTML: https://mlsanity.com/paper/2310.09897/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 61 references that are in the map.
- First posted on arXiv 15 Oct 2023.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [Entailment as Few-Shot Learner](https://mlsanity.com/paper/2104.14690/) (2021)
- [Alzheimer's Dementia Recognition through Spontaneous Speech: The ADReSS Challenge](https://mlsanity.com/paper/2004.06833/) (2020)
- [Improving and Simplifying Pattern Exploiting Training](https://mlsanity.com/paper/2103.11955/) (2021)
- [Towards Learning a Universal Non-Semantic Representation of Speech](https://mlsanity.com/paper/2002.12764/) (2020)
- [Detecting cognitive decline using speech only: The ADReSSo Challenge](https://mlsanity.com/paper/2104.09356/) (2021)
- [To BERT or Not To BERT: Comparing Speech and Language-based Approaches for Alzheimer's Disease Detection](https://mlsanity.com/paper/2008.01551/) (2020)
- [Alzheimer's Dementia Recognition Using Acoustic, Lexical, Disfluency and Speech Pause Features Robust to Noisy Inputs](https://mlsanity.com/paper/2106.15684/) (2021)
- [Detecting Linguistic Characteristics of Alzheimer's Dementia by Interpreting Neural Models](https://mlsanity.com/paper/1804.06440/) (2018)
- [Learning Better Masking for Better Language Model Pre-training](https://mlsanity.com/paper/2208.10806/) (2022)
- [A Review of Automated Speech and Language Features for Assessment of Cognitive and Thought Disorders](https://mlsanity.com/paper/1906.01157/) (2019)
- [Pre-trained Token-replaced Detection Model as Few-shot Learner](https://mlsanity.com/paper/2203.03235/) (2022)

## Built on by (most cited first, 4 of 4)

- [A Systematic Review of NLP for Dementia -- Tasks, Datasets and Opportunities](https://mlsanity.com/paper/2409.19737/) (2024)
- [Profiling Patient Transcript Using Large Language Model Reasoning Augmentation for Alzheimer's Disease Detection](https://mlsanity.com/paper/2409.12541/) (2024)
- [What Do LLMs Know About Alzheimer's Disease? Multi-loss Fine-Tuning and Probing for AD Detection](https://mlsanity.com/paper/2602.11177/) (2026)
- [Fair Cognitive Impairment Detection Through Unlearning](https://mlsanity.com/paper/2606.18571/) (2026)

## Abstract (arXiv)

Dementia is associated with language disorders which impede communication. Here, we automatically learn linguistic disorder patterns by making use of a moderately-sized pre-trained language model and forcing it to focus on reformulated natural language processing (NLP) tasks and associated linguistic patterns. Our experiments show that NLP tasks that encapsulate contextual information and enhance the gradient signal with linguistic patterns benefit performance. We then use the probability estimates from the best model to construct digital linguistic markers measuring the overall quality in communication and the intensity of a variety of language disorders. We investigate how the digital markers characterize dementia speech from a longitudinal perspective. We find that our proposed communication marker is able to robustly and reliably characterize the language of people with dementia, outperforming existing linguistic approaches; and shows external validity via significant correlation with clinical markers of behaviour. Finally, our proposed linguistic disorder markers provide useful insights into gradual language impairment associated with disease progression.