# Big, Bright, or Invisible: A Frozen-Feature Benchmark of 3D CT Foundation Models

arXiv: https://arxiv.org/abs/2608.05960 · HTML: https://mlsanity.com/paper/2608.05960/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 20 references that are in the map.
- First posted on arXiv 6 Aug 2026.

## Builds on (most cited first, 12 of 12)

- [Deep Long-Tailed Learning: A Survey](https://mlsanity.com/paper/2110.04596/) (2021)
- [Generalist Foundation Models from a Multimodal Dataset for 3D Computed Tomography](https://mlsanity.com/paper/2403.17834/) (2024)
- [Machine-Learning-Based Multiple Abnormality Prediction with Large-Scale Chest Computed Tomography Volumes](https://mlsanity.com/paper/2002.04752/) (2020)
- [Vision Foundation Models for Computed Tomography](https://mlsanity.com/paper/2501.09001/) (2025)
- [Pillar-0: A New Frontier for Radiology Foundation Models](https://mlsanity.com/paper/2511.17803/) (2025)
- [Comprehensive language-image pre-training for 3D medical image understanding](https://mlsanity.com/paper/2510.15042/) (2025)
- [Scaling Self-Supervised and Cross-Modal Pretraining for Volumetric CT Transformers](https://mlsanity.com/paper/2511.17209/) (2025)
- [Curia-2: Scaling Self-Supervised Learning for Radiology Foundation Models](https://mlsanity.com/paper/2604.01987/) (2026)
- [Universal CT Representations from Anatomy to Disease Phenotype through Agglomerative Pretraining](https://mlsanity.com/paper/2605.21906/) (2026)
- [Structured Spectral Graph Representation Learning for Multi-label Abnormality Analysis from 3D CT Scans](https://mlsanity.com/paper/2510.10779/) (2025)
- [Learning Robust Visual Features in Computed Tomography Enables Efficient Transfer Learning for Clinical Tasks](https://mlsanity.com/paper/2604.04133/) (2026)
- [LungEvaty: A Scalable, Open-Source Transformer-based Deep Learning Model for Lung Cancer Risk Prediction in LDCT Screening](https://mlsanity.com/paper/2511.20116/) (2025)

## Built on by (most cited first, 1 of 1)

- [The Predictability Bias of Joint-Embedding Predictive Learning in Medical Imaging](https://mlsanity.com/paper/2610.13237/) (2026)

## Abstract (arXiv)

Routine CT interpretation is inherently comprehensive, capturing incidental findings across the entire scan volume. 3D CT foundation models could assist this process by providing generalizable representations of anatomy and pathology. To evaluate their diagnostic breadth, we benchmark ten frozen CT encoders across three cohorts of thoracic CT scans, including an unseen internal clinical dataset, using k-nearest neighbors, zero-shot prompting, and linear probing. We find no universal state-of-the-art, with rankings fluctuating significantly depending on the evaluation context. While models combining fine-grained image tokenization with vision-language alignment generally perform best, a lightweight supervised encoder remains highly competitive, demonstrating that explicit labels can effectively substitute for scale. Crucially, rather than model architecture, we observe that the primary determinant of performance is a physical bottleneck: a finding's detectability scales with its contrast against surrounding tissue and its spatial extent. Through controlled within-organ comparisons, we empirically demonstrate that widespread or high-contrast abnormalities, such as devices and effusions, are reliably recovered. Conversely, small, low-contrast focal lesions remain a persistent challenge across all evaluated encoders. We attribute this to the inherent limitations of globally pooled embeddings, suggesting that accurately representing small, low-contrast structures will require region- or lesion-level pretraining.