# Unifying Acoustic Features and Text with Multimodal LLMs for Neurodegenerative Screening

arXiv: https://arxiv.org/abs/2606.14788 · HTML: https://mlsanity.com/paper/2606.14788/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 10 of its 26 references that are in the map.
- First posted on arXiv 10 Jun 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Medical vision-language models](https://mlsanity.com/mllm/medical-vision-language-models/)

## Builds on (most cited first, 10 of 10)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluation](https://mlsanity.com/paper/2309.14859/) (2023)
- [Detecting Dementia from Speech and Transcripts using Transformers](https://mlsanity.com/paper/2110.14769/) (2021)
- [Generative or Discriminative? Revisiting Text Classification in the Era of Transformers](https://mlsanity.com/paper/2506.12181/) (2025)
- [Detecting Alzheimer's Disease Using Gated Convolutional Neural Network from Audio Data](https://mlsanity.com/paper/1803.11344/) (2018)
- [Alzheimer's disease detection based on large language model prompt engineering](https://mlsanity.com/paper/2501.00861/) (2025)
- [DECT: Harnessing LLM-assisted Fine-Grained Linguistic Knowledge and Label-Switched and Label-Preserved Data Generation for Diagnosis of Alzheimer's Disease](https://mlsanity.com/paper/2502.04394/) (2025)
- [Speech-Based Cognitive Screening: A Systematic Evaluation of LLM Adaptation Strategies](https://mlsanity.com/paper/2509.03525/) (2025)
- [FedKRSO: Communication and Memory Efficient Federated Fine-Tuning of Large Language Models](https://mlsanity.com/paper/2602.03019/) (2026)

## Built on by (most cited first, 1 of 1)

- [Locally Deployable Small Language Models for Emergency Department Decision Support: A Systematic Benchmark of Fine-Tuning Strategies](https://mlsanity.com/paper/2608.10273/) (2026)

## Abstract (arXiv)

Voice-based screening offers a scalable and non-invasive way to assess neurodegenerative diseases such as Alzheimer's disease (AD) and Parkinson's disease (PD), but their staging remains challenging due to the difficulty of integrating heterogeneous data. This paper presents NeurMLLM, an efficient multimodal generative framework for neurodegenerative disease staging. NeurMLLM first encodes the spectrograms and Mel-frequency cepstral coefficients of audio data with vision transformers and projects their representations into the embedding space of a large language model (LLM), where they are concatenated with transcript and demographic instruction tokens as a single unified sequence. The LLM is then instruction-tuned via Low-Rank Adaptation using task prompts to autoregressively predict a constrained label token, enabling a generative classification. By evaluating on the Bridge2AI-Voice dataset for fine-grained staging of AD and PD, we observe that NeurMLLM achieves strong performance, consistently outperforming classical machine learning methods and existing LLM-based approaches. The results show the high potential of multimodal LLMs in neurodegenerative disease staging, improving staging accuracy and supporting accessible deployment.