# Revisiting 2D Foundation Models for Scalable 3D Medical Image Classification arXiv: https://arxiv.org/abs/2512.12887 ยท HTML: https://mlsanity.com/paper/2512.12887/ ## Key facts - Cited by 9 papers in the map (data through 14 Oct 2026). - Builds on 41 of its 69 references that are in the map. - First posted on arXiv 15 Dec 2025. ## Builds on (most cited first, 30 of 41) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023) - [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016) - [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022) - [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023) - [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025) - [EfficientNetV2: Smaller Models and Faster Training](https://mlsanity.com/paper/2104.00298/) (2021) - [Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images](https://mlsanity.com/paper/2201.01266/) (2022) - [Quantifying Attention Flow in Transformers](https://mlsanity.com/paper/2005.00928/) (2020) - [TotalSegmentator: robust segmentation of 104 anatomical structures in CT images](https://mlsanity.com/paper/2208.05868/) (2022) - [MedGemma Technical Report](https://mlsanity.com/paper/2507.05201/) (2025) - [Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis](https://mlsanity.com/paper/2111.14791/) (2021) - [Med3D: Transfer Learning for 3D Medical Image Analysis](https://mlsanity.com/paper/1904.00625/) (2019) - [Co-Scale Conv-Attentional Image Transformers](https://mlsanity.com/paper/2104.06399/) (2021) - [Generalist Foundation Models from a Multimodal Dataset for 3D Computed Tomography](https://mlsanity.com/paper/2403.17834/) (2024) - [DaViT: Dual Attention Vision Transformers](https://mlsanity.com/paper/2204.03645/) (2022) - [3D Self-Supervised Methods for Medical Imaging](https://mlsanity.com/paper/2006.03829/) (2020) - [Merlin: A Computed Tomography Vision-Language Foundation Model and Dataset](https://mlsanity.com/paper/2406.06512/) (2024) - [Virchow: A Million-Slide Digital Pathology Foundation Model](https://mlsanity.com/paper/2309.07778/) (2023) - [VoCo: A Simple-yet-Effective Volume Contrastive Learning Framework for 3D Medical Image Analysis](https://mlsanity.com/paper/2402.17300/) (2024) - [Machine-Learning-Based Multiple Abnormality Prediction with Large-Scale Chest Computed Tomography Volumes](https://mlsanity.com/paper/2002.04752/) (2020) - [Your ViT is Secretly an Image Segmentation Model](https://mlsanity.com/paper/2503.19108/) (2025) - [CrossMoDA 2021 challenge: Benchmark of Cross-Modality Domain Adaptation techniques for Vestibular Schwannoma and Cochlea Segmentation](https://mlsanity.com/paper/2201.02831/) (2022) - [MedImageInsight: An Open-Source Embedding Model for General Domain Medical Imaging](https://mlsanity.com/paper/2410.06542/) (2024) - [MedFMC: A Real-world Dataset and Benchmark For Foundation Model Adaptation in Medical Image Classification](https://mlsanity.com/paper/2306.09579/) (2023) - [Large-Scale 3D Medical Image Pre-training with Geometric Context Priors](https://mlsanity.com/paper/2410.09890/) (2024) - [Evaluating General Purpose Vision Foundation Models for Medical Image Analysis: An Experimental Study of DINOv2 on Radiology Benchmarks](https://mlsanity.com/paper/2312.02366/) (2023) - [Touchstone Benchmark: Are We on the Right Way for Evaluating AI Algorithms for Medical Segmentation?](https://mlsanity.com/paper/2411.03670/) (2024) - [A generalizable 3D framework and model for self-supervised learning in medical imaging](https://mlsanity.com/paper/2501.11755/) (2025) - [SegRap2023: A Benchmark of Organs-at-Risk and Gross Tumor Volume Segmentation for Radiotherapy Planning of Nasopharyngeal Carcinoma](https://mlsanity.com/paper/2312.09576/) (2023) ## Built on by (most cited first, 9 of 9) - [Enhancing Fine-Grained Spatial Grounding in 3D CT Report Generation via Discriminative Guidance](https://mlsanity.com/paper/2604.10437/) (2026) - [Brain-Adapter: A Dual-Stream Vision-Language MIL Framework for Comprehensive 3D CT Diagnosis of Acute Intracranial Pathologies](https://mlsanity.com/paper/2606.23494/) (2026) - [Resolution Meets Reduction: Efficient Visual Context for 3D Radiology Report Generation](https://mlsanity.com/paper/2608.08713/) (2026) - [Specializing Foundation Models via Mixture of Low-Rank Experts for Comprehensive Head CT Analysis](https://mlsanity.com/paper/2603.00675/) (2026) - [PlaneCycle: Training-Free 2D-to-3D Lifting of Foundation Models Without Adapters](https://mlsanity.com/paper/2603.04165/) (2026) - [MultiMedVision: Multi-Modal Medical Vision Framework](https://mlsanity.com/paper/2605.09151/) (2026) - [NV-Reason-CT: 3D Visual Language Model for CT Analysis](https://mlsanity.com/paper/2609.27511/) (2026) - [Natural Image Autoencoder-Based fMRI Representations for Trait and State Prediction](https://mlsanity.com/paper/2609.34167/) (2026) - [Reduce, Then Encode: Multiscale Volumetric Reduction for 2D Foundation Models in Brain MRI](https://mlsanity.com/paper/2609.35405/) (2026) ## Abstract (arXiv) 3D medical image classification is essential for modern clinical workflows. Medical foundation models (FMs) have emerged as a promising approach for scaling to new tasks, yet current research suffers from three critical pitfalls: data-regime bias, suboptimal adaptation, and insufficient task coverage. In this paper, we address these pitfalls and introduce AnyMC3D, a scalable 3D classifier adapted from 2D FMs. Our method scales efficiently to new tasks by adding only lightweight plugins (about 1M parameters per task) on top of a single frozen backbone. This versatile framework also supports multi-view inputs, auxiliary pixel-level supervision, and interpretable heatmap generation. We establish a comprehensive benchmark of 12 tasks covering diverse pathologies, anatomies, and modalities, and systematically analyze state-of-the-art 3D classification techniques. Our analysis reveals key insights: (1) effective adaptation is essential to unlock FM potential, (2) general-purpose FMs can match medical-specific FMs if properly adapted, and (3) 2D-based methods surpass 3D architectures for 3D classification. For the first time, we demonstrate the feasibility of achieving state-of-the-art performance across diverse applications using a single scalable framework (including 1st place in the VLM3D challenge), eliminating the need for separate task-specific models.