# Toward Training-Free Zero-Shot Anomaly Detection in 3D Medical Images: A Batch-Based Approach Using 2D Foundation Models

arXiv: https://arxiv.org/abs/2606.18749 · HTML: https://mlsanity.com/paper/2606.18749/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 55 references that are in the map.
- First posted on arXiv 17 Jun 2026.

## Builds on (most cited first, 21 of 21)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Liver Tumor Segmentation Benchmark (LiTS)](https://mlsanity.com/paper/1901.04056/) (2019)
- [Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation](https://mlsanity.com/paper/1603.05959/) (2016)
- [A Benchmark for Endoluminal Scene Segmentation of Colonoscopy Images](https://mlsanity.com/paper/1612.00799/) (2016)
- [PMC-CLIP: Contrastive Language-Image Pre-training using Biomedical Documents](https://mlsanity.com/paper/2303.07240/) (2023)
- [Automated brain extraction of multi-sequence MRI using artificial neural networks](https://mlsanity.com/paper/1901.11341/) (2019)
- [APRIL-GAN: A Zero-/Few-Shot Anomaly Classification and Segmentation Method for CVPR 2023 VAND Workshop Challenge Tracks 1&2: 1st Place on Zero-shot AD and 4th Place on Few-shot AD](https://mlsanity.com/paper/2305.17382/) (2023)
- [nnU-Net for Brain Tumor Segmentation](https://mlsanity.com/paper/2011.00848/) (2020)
- [Fast Unsupervised Brain Anomaly Detection and Segmentation with Diffusion Models](https://mlsanity.com/paper/2206.03461/) (2022)
- [AbdomenAtlas: A Large-Scale, Detailed-Annotated, & Multi-Center Dataset for Efficient Transfer Learning and Open Algorithmic Benchmarking](https://mlsanity.com/paper/2407.16697/) (2024)
- [MuSc: Zero-Shot Industrial Anomaly Classification and Segmentation with Mutual Scoring of the Unlabeled Images](https://mlsanity.com/paper/2401.16753/) (2024)
- [Patched Diffusion Models for Unsupervised Anomaly Detection in Brain MRI](https://mlsanity.com/paper/2303.03758/) (2023)
- [Rethinking Autoencoders for Medical Anomaly Detection from A Theoretical Perspective](https://mlsanity.com/paper/2403.09303/) (2024)
- [Analysis of the MICCAI Brain Tumor Segmentation -- Metastases (BraTS-METS) 2025 Lighthouse Challenge: Brain Metastasis Segmentation on Pre- and Post-treatment MRI](https://mlsanity.com/paper/2504.12527/) (2025)
- [Unsupervised brain lesion segmentation from MRI using a convolutional autoencoder](https://mlsanity.com/paper/1811.09655/) (2018)
- [Raptor: Scalable Train-Free Embeddings for 3D Medical Volumes Leveraging Pretrained 2D Foundation Models](https://mlsanity.com/paper/2507.08254/) (2025)
- [Feasibility and benefits of joint learning from MRI databases with different brain diseases and modalities for segmentation](https://mlsanity.com/paper/2405.18511/) (2024)
- [cOOpD: Reformulating COPD classification on chest CT scans as anomaly detection using contrastive representations](https://mlsanity.com/paper/2307.07254/) (2023)
- [On the Problem of Consistent Anomalies in Zero-Shot Industrial Anomaly Detection](https://mlsanity.com/paper/2510.10456/) (2025)
- [Training-Free Zero-Shot Anomaly Detection in 3D Brain MRI with 2D Foundation Models](https://mlsanity.com/paper/2602.15315/) (2026)
- [VoxCor: Training-Free Volumetric Features for Multimodal Voxel Correspondence](https://mlsanity.com/paper/2605.13798/) (2026)

## Abstract (arXiv)

Zero-shot anomaly detection (ZSAD) is attractive for medical imaging because clinical systems must handle heterogeneous acquisition protocols, changing patient populations, and pathologies for which annotated training data may be unavailable. Most existing zero-shot anomaly detection methods are designed for 2D images, and their direct extension to 3D medical volumes is limited by the scarcity of large-scale volumetric foundation models or by the difficulty of utilizing volumetric context. We propose CS3F, a training-free batch-based framework for ZSAD in 3D medical images using 2D foundation models. Each volume is decomposed along multiple anatomical axes and encoded slice-wise by a 2D vision transformer. These are then converted into localized volumetric tokens by pooling neighboring slice features. Anomaly scores are obtained from cross-subject mutual similarity: tokens that lack close analogues in other subjects are assigned higher anomaly scores. To reduce the attenuation of focal lesion signals caused by depth pooling, we introduce a coarse-to-fine tokenization strategy that enables fine-resolution volumetric scoring without exhaustive matching. CS3F is evaluated on brain MRI across metastases, glioma, and stroke, as well as validated on lung CT to test generalizability beyond atlas-aligned brain MRI. The results show that frozen 2D foundation models can support anomaly localization in 3D medical images, and that the benefit of fine tokenization depends strongly on lesion contrast and imaging modality.