# Diagnosing Alzheimer's Disease using Early-Late Multimodal Data Fusion with Jacobian Maps

arXiv: https://arxiv.org/abs/2310.16936 · HTML: https://mlsanity.com/paper/2310.16936/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 32 references that are in the map.
- First posted on arXiv 25 Oct 2023.

## Builds on (most cited first, 5 of 5)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Early Prediction of Alzheimer's Disease Dementia Based on Baseline Hippocampal MRI and 1-Year Follow-Up Cognitive Measures Using Deep Recurrent Neural Networks](https://mlsanity.com/paper/1901.01451/) (2019)
- [Preclinical Stage Alzheimer's Disease Detection Using Magnetic Resonance Image Scans](https://mlsanity.com/paper/2011.14139/) (2020)
- [Input Agnostic Deep Learning for Alzheimer's Disease Classification Using Multimodal MRI Images](https://mlsanity.com/paper/2107.08673/) (2021)
- [A Brain-Computer Interface Augmented Reality Framework with Auto-Adaptive SSVEP Recognition](https://mlsanity.com/paper/2308.06401/) (2023)

## Built on by (most cited first, 5 of 5)

- [A review of deep learning-based information fusion techniques for multimodal medical image classification](https://mlsanity.com/paper/2404.15022/) (2024)
- [Unmasking Dementia Detection by Masking Input Gradients: A JSM Approach to Model Interpretability and Precision](https://mlsanity.com/paper/2402.16008/) (2024)
- [Cross-Attention Fusion of MRI and Jacobian Maps for Alzheimer's Disease Diagnosis](https://mlsanity.com/paper/2503.00586/) (2025)
- [Efficient Brain Imaging Analysis for Alzheimer's and Dementia Detection Using Convolution-Derivative Operations](https://mlsanity.com/paper/2411.13490/) (2024)
- [Unlocking Neural Transparency: Jacobian Maps for Explainable AI in Alzheimer's Detection](https://mlsanity.com/paper/2504.03230/) (2025)

## Abstract (arXiv)

Alzheimer's disease (AD) is a prevalent and debilitating neurodegenerative disorder impacting a large aging population. Detecting AD in all its presymptomatic and symptomatic stages is crucial for early intervention and treatment. An active research direction is to explore machine learning methods that harness multimodal data fusion to outperform human inspection of medical scans. However, existing multimodal fusion models have limitations, including redundant computation, complex architecture, and simplistic handling of missing data. Moreover, the preprocessing pipelines of medical scans remain inadequately detailed and are seldom optimized for individual subjects. In this paper, we propose an efficient early-late fusion (ELF) approach, which leverages a convolutional neural network for automated feature extraction and random forests for their competitive performance on small datasets. Additionally, we introduce a robust preprocessing pipeline that adapts to the unique characteristics of individual subjects and makes use of whole brain images rather than slices or patches. Moreover, to tackle the challenge of detecting subtle changes in brain volume, we transform images into the Jacobian domain (JD) to enhance both accuracy and robustness in our classification. Using MRI and CT images from the OASIS-3 dataset, our experiments demonstrate the effectiveness of the ELF approach in classifying AD into four stages with an accuracy of 97.19%.