# A Longitudinal Multi-modal Dataset for Dementia Monitoring and Diagnosis

arXiv: https://arxiv.org/abs/2109.01537 · HTML: https://mlsanity.com/paper/2109.01537/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 70 references that are in the map.
- First posted on arXiv 3 Sep 2021.

## Builds on (most cited first, 9 of 9)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Alzheimer's Dementia Recognition through Spontaneous Speech: The ADReSS Challenge](https://mlsanity.com/paper/2004.06833/) (2020)
- [Towards Learning a Universal Non-Semantic Representation of Speech](https://mlsanity.com/paper/2002.12764/) (2020)
- [Detecting cognitive decline using speech only: The ADReSSo Challenge](https://mlsanity.com/paper/2104.09356/) (2021)
- [Cultural Shift or Linguistic Drift? Comparing Two Computational Measures of Semantic Change](https://mlsanity.com/paper/1606.02821/) (2016)
- [The Second DIHARD Diarization Challenge: Dataset, task, and baselines](https://mlsanity.com/paper/1906.07839/) (2019)
- [Detecting Linguistic Characteristics of Alzheimer's Dementia by Interpreting Neural Models](https://mlsanity.com/paper/1804.06440/) (2018)
- [End-to-end Domain-Adversarial Voice Activity Detection](https://mlsanity.com/paper/1910.10655/) (2019)
- [A Review of Automated Speech and Language Features for Assessment of Cognitive and Thought Disorders](https://mlsanity.com/paper/1906.01157/) (2019)

## Built on by (most cited first, 2 of 2)

- [Evaluating Picture Description Speech for Dementia Detection using Image-text Alignment](https://mlsanity.com/paper/2308.07933/) (2023)
- [Temporal-Aware Iterative Speech Model for Dementia Detection](https://mlsanity.com/paper/2510.00030/) (2025)

## Abstract (arXiv)

Dementia affects cognitive functions of adults, including memory, language, and behaviour. Standard diagnostic biomarkers such as MRI are costly, whilst neuropsychological tests suffer from sensitivity issues in detecting dementia onset. The analysis of speech and language has emerged as a promising and non-intrusive technology to diagnose and monitor dementia. Currently, most work in this direction ignores the multi-modal nature of human communication and interactive aspects of everyday conversational interaction. Moreover, most studies ignore changes in cognitive status over time due to the lack of consistent longitudinal data. Here we introduce a novel fine-grained longitudinal multi-modal corpus collected in a natural setting from healthy controls and people with dementia over two phases, each spanning 28 sessions. The corpus consists of spoken conversations, a subset of which are transcribed, as well as typed and written thoughts and associated extra-linguistic information such as pen strokes and keystrokes. We present the data collection process and describe the corpus in detail. Furthermore, we establish baselines for capturing longitudinal changes in language across different modalities for two cohorts, healthy controls and people with dementia, outlining future research directions enabled by the corpus.