mlsanityData to 7 Oct 2026

CogniVoice: Multimodal and Multilingual Fusion Networks for Mild Cognitive Impairment Assessment from Spontaneous Speech

arXiv 2407.13660 ↗First posted 18 Jul 2024cs.LG cs.SD eess.AScited by 8

Not on the map. Why?

Abstract (arXiv)

Mild Cognitive Impairment (MCI) is a medical condition characterized by noticeable declines in memory and cognitive abilities, potentially affecting individual's daily activities. In this paper, we introduce CogniVoice, a novel multilingual and multimodal framework to detect MCI and estimate Mini-Mental State Examination (MMSE) scores by analyzing speech data and its textual transcriptions. The key component of CogniVoice is an ensemble multimodal and multilingual network based on ``Product of Experts'' that mitigates reliance on shortcut solutions. Using a comprehensive dataset containing both English and Chinese languages from TAUKADIAL challenge, CogniVoice outperforms the best performing baseline model on MCI classification and MMSE regression tasks by 2.8 and 4.1 points in F1 and RMSE respectively, and can effectively reduce the performance gap across different language groups by 0.7 points in F1.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 16 of its 32 references that are in the map. First posted on arXiv 18 Jul 2024.

Builds on

16 of 32 references.

References · 16

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  4. 2021AST: Audio Spectrogram Transformercited by 634
  5. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  6. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 496
  7. 2019Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biasescited by 279
  8. 2019RUBi: Reducing Unimodal Biases in Visual Question Answeringcited by 183
  9. 2019End-to-End Bias Mitigation by Modelling Biases in Corporacited by 127
  10. 2020To BERT or Not To BERT: Comparing Speech and Language-based Approaches for Alzheimer's Disease Detectioncited by 49
  11. 2023Toward Joint Language Modeling for Speech Units and Textcited by 27
  12. 2024Connected Speech-Based Cognitive Assessment in Chinese and Englishcited by 13
4 more
  1. 2023Cross-lingual Alzheimer's Disease detection based on paralinguistic and pre-trained featurescited by 8
  2. 2017Enriching Complex Networks with Word Embeddings for Detecting Mild Cognitive Impairment from Speech Transcriptscited by 7
  3. 2023Cross-Lingual Transfer Learning for Alzheimer's Detection From Spontaneous Speechcited by 6
  4. 2023A Digital Language Coherence Marker for Monitoring Dementiacited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 2.

8 papers.

  1. 2025Speech Unlearningcited by 7
  2. 2025EqualizeIR: Mitigating Linguistic Biases in Retrieval Modelscited by 5
  3. 2025Do Students Debias Like Teachers? On the Distillability of Bias Mitigation Methodscited by 2
  4. 2026False positive bias in AI-powered speech-based cognitive screening for multilingual English speakers in the UKcited by 2
  5. 2025Unveil Multi-Picture Descriptions for Multilingual Mild Cognitive Impairment Detection via Contrastive Learningcited by 1
  6. 2025Enhancing and Exploring Mild Cognitive Impairment Detection with W2V-BERT-2.0cited by 0
  7. 2025FairFlow: Mitigating Dataset Biases through Undecided Learningcited by 0
  8. 2026Fair Cognitive Impairment Detection Through Unlearningcited by 0