# Alzheimer's Disease Diagnosis using a Multimodal Approach with 3D MRI and PET

arXiv: https://arxiv.org/abs/2606.20037 · HTML: https://mlsanity.com/paper/2606.20037/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 30 references that are in the map.
- First posted on arXiv 18 Jun 2026.

## Builds on (most cited first, 9 of 9)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Multimodal Attention-based Deep Learning for Alzheimer's Disease Diagnosis](https://mlsanity.com/paper/2206.08826/) (2022)
- [Predicting Alzheimer's disease: a neuroimaging study with 3D convolutional neural networks](https://mlsanity.com/paper/1502.02506/) (2015)
- [Diagnosis of Alzheimer's Disease via Multi-modality 3D Convolutional Neural Network](https://mlsanity.com/paper/1902.09904/) (2019)
- [Multimodal and Multiscale Deep Neural Networks for the Early Diagnosis of Alzheimer's Disease using structural MR and FDG-PET images](https://mlsanity.com/paper/1710.04782/) (2017)
- [Multimodal Unified Attention Networks for Vision-and-Language Interactions](https://mlsanity.com/paper/1908.04107/) (2019)
- [Classification of Alzheimer's Disease using fMRI Data and Deep Learning Convolutional Neural Networks](https://mlsanity.com/paper/1603.08631/) (2016)

## Abstract (arXiv)

Alzheimer's disease (AD) is an irreversible neurodegenerative disorder and a leading cause of death worldwide. Early diagnosis plays an important part especially at the Mild Cognitive Impairment stage, where timely intervention can help slow its progression before it advances to AD. Neuroimaging data, like Magnetic Resonance Imaging (MRI) and Positron Emission Tomography (PET) scans, can help detect brain changes early by providing structural and functional brain changes related to the disease. Yet, many multimodal models still fuse MRI and PET with static concatenation and apply identical computation to all subjects, which limits robustness to patient/site heterogeneity and can waste computation. To address these limitations, we present the first study of combining 3D convolutional feature extractors with three fusion strategies - concatenation, Gated Multimodal Unit (GMU), and gated self-attention - and a sparsely gated Mixture-of-Experts (MoE) classifier that performs input-adaptive routing, activating only the most informative experts per case. Finally, we utilize Grad-CAM to visualize disease-related regions, ensuring model interpretability. Experiments are performed across three binary classification tasks (NC vs. MCI, MCI vs. AD, and NC vs. AD). Results show that GMU achieves accuracies of 80.46 % (NC vs. MCI) and 95.47 % (NC vs. AD), while gated self-attention attains 82.08 % on MCI vs. AD. Ablations show that removing the MoE consistently degrades accuracy across all tasks. These findings underscore the value of input-adaptive, multimodal modeling for AD diagnosis by leveraging the complementary nature of MRI and PET.