# DinoAtten3D: Slice-Level Attention Aggregation of DinoV2 for 3D Brain MRI Anomaly Classification

arXiv: https://arxiv.org/abs/2509.12512 · HTML: https://mlsanity.com/paper/2509.12512/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 45 references that are in the map.
- First posted on arXiv 15 Sep 2025.

## Builds on (most cited first, 28 of 28)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images](https://mlsanity.com/paper/2004.09666/) (2020)
- [BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs](https://mlsanity.com/paper/2303.00915/) (2023)
- [Segment Anything in Medical Images](https://mlsanity.com/paper/2304.12306/) (2023)
- [Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning](https://mlsanity.com/paper/1602.03409/) (2016)
- [Multiple Instance Learning: A Survey of Problem Characteristics and Applications](https://mlsanity.com/paper/1612.03365/) (2016)
- [Brain Tumor Segmentation with Deep Neural Networks](https://mlsanity.com/paper/1505.03540/) (2015)
- [On the Challenges and Perspectives of Foundation Models for Medical Image Analysis](https://mlsanity.com/paper/2306.05705/) (2023)
- [Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models](https://mlsanity.com/paper/2304.12526/) (2023)
- [CNN-based Segmentation of Medical Imaging Data](https://mlsanity.com/paper/1701.03056/) (2017)
- [Are Natural Domain Foundation Models Useful for Medical Image Classification?](https://mlsanity.com/paper/2310.19522/) (2023)
- [Anomaly Detection in Medical Imaging -- A Mini Review](https://mlsanity.com/paper/2108.11986/) (2021)
- [Evaluating General Purpose Vision Foundation Models for Medical Image Analysis: An Experimental Study of DINOv2 on Radiology Benchmarks](https://mlsanity.com/paper/2312.02366/) (2023)
- [Few-shot Adaptation of Multi-modal Foundation Models: A Survey](https://mlsanity.com/paper/2401.01736/) (2024)
- [Scaling Pre-training to One Hundred Billion Data for Vision Language Models](https://mlsanity.com/paper/2502.07617/) (2025)
- [How much is a noisy image worth? Data Scaling Laws for Ambient Diffusion](https://mlsanity.com/paper/2411.02780/) (2024)
- [Pruning then Reweighting: Towards Data-Efficient Training of Diffusion Models](https://mlsanity.com/paper/2409.19128/) (2024)
- [Overcoming the limitations of patch-based learning to detect cancer in whole slide images](https://mlsanity.com/paper/2012.00617/) (2020)
- [Brainomaly: Unsupervised Neurologic Disease Detection Utilizing Unannotated T1-weighted Brain MR Images](https://mlsanity.com/paper/2302.09200/) (2023)
- [Automatically Designing CNN Architectures for Medical Image Segmentation](https://mlsanity.com/paper/1807.07663/) (2018)
- [HealthyGAN: Learning from Unannotated Medical Images to Detect Anomalies Associated with Human Disease](https://mlsanity.com/paper/2209.01822/) (2022)
- [SC-MIL: Supervised Contrastive Multiple Instance Learning for Imbalanced Classification in Pathology](https://mlsanity.com/paper/2303.13405/) (2023)
- [Diffusion Models for Unsupervised Anomaly Detection in Fetal Brain Ultrasound](https://mlsanity.com/paper/2407.15119/) (2024)
- [Efficient Feature Extraction and Classification Architecture for MRI-Based Brain Tumor Detection and Localization](https://mlsanity.com/paper/2410.22619/) (2024)

## Built on by (most cited first, 3 of 3)

- [Brain-Adapter: A Dual-Stream Vision-Language MIL Framework for Comprehensive 3D CT Diagnosis of Acute Intracranial Pathologies](https://mlsanity.com/paper/2606.23494/) (2026)
- [Multimodal Visual Surrogate Compression for Alzheimer's Disease Classification](https://mlsanity.com/paper/2601.21673/) (2026)
- [Reduce, Then Encode: Multiscale Volumetric Reduction for 2D Foundation Models in Brain MRI](https://mlsanity.com/paper/2609.35405/) (2026)

## Abstract (arXiv)

Anomaly detection and classification in medical imaging are critical for early diagnosis but remain challenging due to limited annotated data, class imbalance, and the high cost of expert labeling. Emerging vision foundation models such as DINOv2, pretrained on extensive, unlabeled datasets, offer generalized representations that can potentially alleviate these limitations. In this study, we propose an attention-based global aggregation framework tailored specifically for 3D medical image anomaly classification. Leveraging the self-supervised DINOv2 model as a pretrained feature extractor, our method processes individual 2D axial slices of brain MRIs, assigning adaptive slice-level importance weights through a soft attention mechanism. To further address data scarcity, we employ a composite loss function combining supervised contrastive learning with class-variance regularization, enhancing inter-class separability and intra-class consistency. We validate our framework on the ADNI dataset and an institutional multi-class headache cohort, demonstrating strong anomaly classification performance despite limited data availability and significant class imbalance. Our results highlight the efficacy of utilizing pretrained 2D foundation models combined with attention-based slice aggregation for robust volumetric anomaly detection in medical imaging. Our implementation is publicly available at https://github.com/Rafsani/DinoAtten3D.git.