# The INESC-ID Multi-Modal System for the ADReSS 2020 Challenge

arXiv: https://arxiv.org/abs/2005.14646 · HTML: https://mlsanity.com/paper/2005.14646/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 May 2020.

## Built on by (most cited first, 15 of 15)

- [Comparing Acoustic-based Approaches for Alzheimer's Disease Detection](https://mlsanity.com/paper/2106.01555/) (2021)
- [Explainable Identification of Dementia from Transcripts using Transformer Networks](https://mlsanity.com/paper/2109.06980/) (2021)
- [Detecting Dementia from Speech and Transcripts using Transformers](https://mlsanity.com/paper/2110.14769/) (2021)
- [Exploring linguistic feature and model combination for speech recognition based automatic AD detection](https://mlsanity.com/paper/2206.13758/) (2022)
- [Learning De-identified Representations of Prosody from Raw Audio](https://mlsanity.com/paper/2107.08248/) (2021)
- [A Multimodal Approach for Dementia Detection from Spontaneous Speech with Tensor Fusion Layer](https://mlsanity.com/paper/2211.04368/) (2022)
- [Context-aware attention layers coupled with optimal transport domain adaptation and multimodal fusion methods for recognizing dementia from spontaneous speech](https://mlsanity.com/paper/2305.16406/) (2023)
- [Alzheimer Disease Classification through ASR-based Transcriptions: Exploring the Impact of Punctuation and Pauses](https://mlsanity.com/paper/2306.03443/) (2023)
- [Alzheimers Dementia Detection using Acoustic & Linguistic features and Pre-Trained BERT](https://mlsanity.com/paper/2109.11010/) (2021)
- [Neural Architecture Search with Multimodal Fusion Methods for Diagnosing Dementia](https://mlsanity.com/paper/2302.05894/) (2023)
- [A Systematic Review of NLP for Dementia -- Tasks, Datasets and Opportunities](https://mlsanity.com/paper/2409.19737/) (2024)
- [Linguistic-Based Mild Cognitive Impairment Detection Using Informative Loss](https://mlsanity.com/paper/2402.01690/) (2024)
- [Deep Insights into Cognitive Decline: A Survey of Leveraging Non-Intrusive Modalities with Deep Learning Techniques](https://mlsanity.com/paper/2410.18972/) (2024)
- [Acoustic-Linguistic Features for Modeling Neurological Task Score in Alzheimer's](https://mlsanity.com/paper/2209.06085/) (2022)
- [Dementia classification from spontaneous speech using wrapper-based feature selection](https://mlsanity.com/paper/2502.03484/) (2025)

## Abstract (arXiv)

This paper describes a multi-modal approach for the automatic detection of Alzheimer's disease proposed in the context of the INESC-ID Human Language Technology Laboratory participation in the ADReSS 2020 challenge. Our classification framework takes advantage of both acoustic and textual feature embeddings, which are extracted independently and later combined. Speech signals are encoded into acoustic features using DNN speaker embeddings extracted from pre-trained models. For textual input, contextual embedding vectors are first extracted using an English Bert model and then used either to directly compute sentence embeddings or to feed a bidirectional LSTM-RNNs with attention. Finally, an SVM classifier with linear kernel is used for the individual evaluation of the three systems. Our best system, based on the combination of linguistic and acoustic information, attained a classification accuracy of 81.25%. Results have shown the importance of linguistic features in the classification of Alzheimer's Disease, which outperforms the acoustic ones in terms of accuracy. Early stage features fusion did not provide additional improvements, confirming that the discriminant ability conveyed by speech in this case is smooth out by linguistic data.