# Exploiting Multi-Modal Features From Pre-trained Networks for Alzheimer's Dementia Recognition

arXiv: https://arxiv.org/abs/2009.04070 · HTML: https://mlsanity.com/paper/2009.04070/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Sep 2020.

## Built on by (most cited first, 20 of 20)

- [Detecting Dementia from Speech and Transcripts using Transformers](https://mlsanity.com/paper/2110.14769/) (2021)
- [Leveraging Pretrained Representations with Task-related Keywords for Alzheimer's Disease Detection](https://mlsanity.com/paper/2303.08019/) (2023)
- [Exploring linguistic feature and model combination for speech recognition based automatic AD detection](https://mlsanity.com/paper/2206.13758/) (2022)
- [LLMCARE: early detection of cognitive impairment via transformer models enhanced by LLM-generated synthetic data](https://mlsanity.com/paper/2508.10027/) (2025)
- [A Multimodal Approach for Dementia Detection from Spontaneous Speech with Tensor Fusion Layer](https://mlsanity.com/paper/2211.04368/) (2022)
- [Context-aware attention layers coupled with optimal transport domain adaptation and multimodal fusion methods for recognizing dementia from spontaneous speech](https://mlsanity.com/paper/2305.16406/) (2023)
- [Alzheimer Disease Classification through ASR-based Transcriptions: Exploring the Impact of Punctuation and Pauses](https://mlsanity.com/paper/2306.03443/) (2023)
- [A Review of Deep Learning Approaches for Non-Invasive Cognitive Impairment Detection](https://mlsanity.com/paper/2410.19898/) (2024)
- [Alzheimers Dementia Detection using Acoustic & Linguistic features and Pre-Trained BERT](https://mlsanity.com/paper/2109.11010/) (2021)
- [Evaluating Picture Description Speech for Dementia Detection using Image-text Alignment](https://mlsanity.com/paper/2308.07933/) (2023)
- [Transferring speech-generic and depression-specific knowledge for Alzheimer's disease detection](https://mlsanity.com/paper/2310.04358/) (2023)
- [A Systematic Review of NLP for Dementia -- Tasks, Datasets and Opportunities](https://mlsanity.com/paper/2409.19737/) (2024)
- [Swin-BERT: A Feature Fusion System designed for Speech-based Alzheimer's Dementia Detection](https://mlsanity.com/paper/2410.07277/) (2024)
- [Not All Errors Are Equal: Investigation of Speech Recognition Errors in Alzheimer's Disease Detection](https://mlsanity.com/paper/2412.06332/) (2024)
- [On the Within-class Variation Issue in Alzheimer's Disease Detection](https://mlsanity.com/paper/2409.16322/) (2024)
- [Deep Insights into Cognitive Decline: A Survey of Leveraging Non-Intrusive Modalities with Deep Learning Techniques](https://mlsanity.com/paper/2410.18972/) (2024)
- [Dementia classification from spontaneous speech using wrapper-based feature selection](https://mlsanity.com/paper/2502.03484/) (2025)
- [Agentic Cognitive Profiling: Realigning Automated Alzheimer's Disease Detection with Clinical Construct Validity](https://mlsanity.com/paper/2603.17392/) (2026)
- [Toward Generalizable Cognitive Impairment Detection with Speech-Based Multimodal Large Language Models](https://mlsanity.com/paper/2607.21496/) (2026)
- [Explainable and Generalisable LLM-based Cognitive Decline Detection with Spontaneous Speech](https://mlsanity.com/paper/2609.34217/) (2026)

## Abstract (arXiv)

Collecting and accessing a large amount of medical data is very time-consuming and laborious, not only because it is difficult to find specific patients but also because it is required to resolve the confidentiality of a patient's medical records. On the other hand, there are deep learning models, trained on easily collectible, large scale datasets such as Youtube or Wikipedia, offering useful representations. It could therefore be very advantageous to utilize the features from these pre-trained networks for handling a small amount of data at hand. In this work, we exploit various multi-modal features extracted from pre-trained networks to recognize Alzheimer's Dementia using a neural network, with a small dataset provided by the ADReSS Challenge at INTERSPEECH 2020. The challenge regards to discern patients suspicious of Alzheimer's Dementia by providing acoustic and textual data. With the multi-modal features, we modify a Convolutional Recurrent Neural Network based structure to perform classification and regression tasks simultaneously and is capable of computing conversations with variable lengths. Our test results surpass baseline's accuracy by 18.75%, and our validation result for the regression task shows the possibility of classifying 4 classes of cognitive impairment with an accuracy of 78.70%.