# Multimodal Visual Surrogate Compression for Alzheimer's Disease Classification

arXiv: https://arxiv.org/abs/2601.21673 · HTML: https://mlsanity.com/paper/2601.21673/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 47 references that are in the map.
- First posted on arXiv 29 Jan 2026.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day](https://mlsanity.com/paper/2306.00890/) (2023)
- [Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2301.08243/) (2023)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders](https://mlsanity.com/paper/2301.00808/) (2023)
- [MedCLIP: Contrastive Learning from Unpaired Medical Images and Text](https://mlsanity.com/paper/2210.10163/) (2022)
- [Making the Most of Text Semantics to Improve Biomedical Vision--Language Processing](https://mlsanity.com/paper/2204.09817/) (2022)
- [SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining](https://mlsanity.com/paper/2107.09559/) (2021)
- [Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing](https://mlsanity.com/paper/2301.04558/) (2023)
- [SynthStrip: Skull-Stripping for Any Brain Image](https://mlsanity.com/paper/2203.09974/) (2022)
- [Merlin: A Computed Tomography Vision-Language Foundation Model and Dataset](https://mlsanity.com/paper/2406.06512/) (2024)
- [VISTA3D: A Unified Segmentation Foundation Model For 3D Medical Imaging](https://mlsanity.com/paper/2406.05285/) (2024)
- [Enhancing Spatiotemporal Disease Progression Models via Latent Diffusion and Prior Knowledge](https://mlsanity.com/paper/2405.03328/) (2024)
- [PyTorchVideo: A Deep Learning Library for Video Understanding](https://mlsanity.com/paper/2111.09887/) (2021)
- [Brain Harmony: A Multimodal Foundation Model Unifying Morphology and Function into 1D Tokens](https://mlsanity.com/paper/2509.24693/) (2025)
- [Raptor: Scalable Train-Free Embeddings for 3D Medical Volumes Leveraging Pretrained 2D Foundation Models](https://mlsanity.com/paper/2507.08254/) (2025)
- [Longformer: Longitudinal Transformer for Alzheimer's Disease Classification with Structural MRIs](https://mlsanity.com/paper/2302.00901/) (2023)
- [MedUnifier: Unifying Vision-and-Language Pre-training on Medical Data with Vision Generation Task using Discrete Visual Representations](https://mlsanity.com/paper/2503.01019/) (2025)
- [Towards Generalisable Foundation Models for Brain MRI](https://mlsanity.com/paper/2510.23415/) (2025)
- [AXIAL: Attention-based eXplainability for Interpretable Alzheimer's Localized Diagnosis using 2D CNNs on 3D MRI brain scans](https://mlsanity.com/paper/2407.02418/) (2024)
- [DinoAtten3D: Slice-Level Attention Aggregation of DinoV2 for 3D Brain MRI Anomaly Classification](https://mlsanity.com/paper/2509.12512/) (2025)
- [TomoGraphView: 3D Medical Image Classification with Omnidirectional Slice Representations and Graph Neural Networks](https://mlsanity.com/paper/2511.09605/) (2025)

## Abstract (arXiv)

High-dimensional structural MRI (sMRI) images are widely used for Alzheimer's Disease (AD) diagnosis. Most existing methods for sMRI representation learning rely on 3D architectures (e.g., 3D CNNs), slice-wise feature extraction with late aggregation, or apply training-free feature extractions using 2D foundation models (e.g., DINO). However, these three paradigms suffer from high computational cost, loss of cross-slice relations, and limited ability to extract discriminative features, respectively. To address these challenges, we propose Multimodal Visual Surrogate Compression (MVSC). It learns to compress and adapt large 3D sMRI volumes into compact 2D features, termed as visual surrogates, which are better aligned with frozen 2D foundation models to extract powerful representations for final AD classification. MVSC has two key components: a Volume Context Encoder that captures global cross-slice context under textual guidance, and an Adaptive Slice Fusion module that aggregates slice-level information in a text-enhanced, patch-wise manner. Extensive experiments on three large-scale Alzheimer's disease benchmarks demonstrate our MVSC performs favourably on both binary and multi-class classification tasks compared against state-of-the-art methods.