mlsanityData to 14 Oct 2026

Pillar-0: A New Frontier for Radiology Foundation Models

arXiv 2511.17803 ↗First posted 21 Nov 2025cs.CV cs.AIcited by 25

Not on the map. Why?

Abstract (arXiv)

Radiology plays an integral role in modern medicine, yet rising imaging volumes have far outpaced workforce growth. Foundation models offer a path toward assisting with the full spectrum of radiology tasks, but existing medical models remain limited: they process volumetric CT and MRI as low-fidelity 2D slices, discard critical grayscale contrast information, and lack evaluation frameworks that reflect real clinical practice. We introduce Pillar-0, a radiology foundation model pretrained on 42,990 abdomen-pelvis CTs, 86,411 chest CTs, 14,348 head CTs, and 11,543 breast MRIs from a large academic center, together with RATE, a scalable framework that extracts structured labels for 366 radiologic findings with near-perfect accuracy using LLMs. Across internal test sets of 14,230 abdomen-pelvis CTs, 10,646 chest CTs, 4,906 head CTs, and 1,585 breast MRIs, Pillar-0 establishes a new performance frontier, achieving mean AUROCs of 86.4, 88.0, 90.1, and 82.9, outperforming MedGemma (Google), MedImageInsight (Microsoft), Lingshu (Alibaba), and Merlin (Stanford) by 7.8-15.8 AUROC points and ranking best in 87.2% (319/366) tasks. Pillar-0 similarly outperforms all baselines in an external validation on the Stanford Abdominal CT dataset, including Merlin (82.2 vs 80.6 AUROC). Pillar-0 extends to tasks beyond its pretraining, such as long-horizon lung cancer risk prediction, where it improves upon the state-of-the-art Sybil by 3.0 C-index points on NLST, and generalizes with gains of 5.9 (MGH) and 1.9 (CGMH). In brain hemorrhage detection, Pillar-0 obtained a >95 AUROC when using only 1/20th of the data of the next most sample efficient baseline. Pillar-0 and RATE together provide an open, clinically rigorous foundation for building high-performance radiology systems, enabling applications that were previously infeasible due to computational, data, and evaluation constraints.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Builds on 25 of its 53 references that are in the map. First posted on arXiv 21 Nov 2025.

Builds on

25 of 53 references.

References · 25

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2020End-to-End Object Detection with Transformerscited by 7,414
  5. 2025Qwen2.5-VL Technical Reportcited by 5,587
  6. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  7. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  8. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
  9. 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 520
  10. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 449
  11. 2025MedGemma Technical Reportcited by 436
  12. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 338
13 more
  1. 2019Med3D: Transfer Learning for 3D Medical Image Analysiscited by 288
  2. 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoningcited by 230
  3. 2025MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understandingcited by 224
  4. 2022Unified Contrastive Learning in Image-Text-Label Spacecited by 194
  5. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLMcited by 184
  6. 2022DaViT: Dual Attention Vision Transformerscited by 175
  7. 2022OmniVL:One Foundation Model for Image-Language and Video-Language Taskscited by 143
  8. 2023Swin3D: A Pretrained Transformer Backbone for 3D Indoor Scene Understandingcited by 113
  9. 2024Merlin: A Computed Tomography Vision-Language Foundation Model and Datasetcited by 102
  10. 2024MedImageInsight: An Open-Source Embedding Model for General Domain Medical Imagingcited by 46
  11. 2025Atlas: Multi-Scale Attention Improves Long Context Image Modelingcited by 4
  12. 2025Automated Real-time Assessment of Intracranial Hemorrhage Detection AI Using an Ensembled Monitoring Model (EMM)cited by 3
  13. 2025Radiology's Last Exam (RadLE): Benchmarking Frontier Multimodal AI Against Human Experts and a Taxonomy of Visual Reasoning Errors in Radiologycited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 25.

25 papers.

Sort:

By area:

  1. 2026Curia-2: Scaling Self-Supervised Learning for Radiology Foundation Modelscited by 6
  2. 2026TotalFM: An Organ-Separated 3D-CT Foundation Model Leveraging Large-Scale Routine Clinical Radiology Datacited by 4
  3. 2026Jolia: Concept-Level Vision-Language Alignment for 3D CT Contrastive Learningcited by 4
  4. 2026Learning to Read Where to Look: Disease-Aware Vision-Language Pretraining for 3D CTcited by 3
  5. 2026Enhancing Fine-Grained Spatial Grounding in 3D CT Report Generation via Discriminative Guidancecited by 3
  6. 2026GreenRFM: Learning a resource-efficient radiology vision-language foundation model via supervision-centric pre-trainingcited by 2
  7. 2026MedPruner: Training-Free Hierarchical Token Pruning for Efficient 3D Medical Image Understanding in Vision-Language Modelscited by 1
  8. 2026Pan-FM: A Pan-Organ Foundation Model with Saliency-Guided Masking for Missing Robustnesscited by 1
  9. 2026CT-IDP: Segmentation-Derived Quantitative Phenotypes for Interpretable Abdominal CT Disease Classificationcited by 1
  10. 2026JANUS: Anatomy-Conditioned Gating for Robust CT Triage Under Distribution Shiftcited by 1
  11. 2026ORACLE-CT: Anatomy-Aware Support Pooling for CT Classificationcited by 1
  12. 2026Open-Ended CT Volume Segmentation with Weak Supervision from Languagecited by 1
13 more
  1. 2026Big, Bright, or Invisible: A Frozen-Feature Benchmark of 3D CT Foundation Modelscited by 1
  2. 2026Specializing Foundation Models via Mixture of Low-Rank Experts for Comprehensive Head CT Analysiscited by 0
  3. 2026Blinded Radiologist and LLM-Based Evaluation of LLM-Generated Japanese Translations of Chest CT Reports: Comparative Studycited by 0
  4. 2026UCSF-PDGM-VQA: Visual Question Answering dataset for brain tumor MRI interpretationcited by 0
  5. 2026MedSyn2: Flexible Control of 3D CT Generation via Text and Semantically-Defined Segmentation Promptscited by 0
  6. 2026Stateful Visual Encoders for Vision-Language Modelscited by 0
  7. 2026When is 3D Worth It? A Resource-Performance Frontier for CNNs and Transformers in Lung CTcited by 0
  8. 2026A Multi-Center Benchmark for Abdominal Disease Diagnosis and Report Generation from Non-Contrast CTcited by 0
  9. 2026Curia-MAE: Multi-Modal Multi-Anatomy MAE Pre-Training for 3D Medical Image Segmentationcited by 0
  10. 2026Longitudinal 3D Foundation Modeling for Neoadjuvant Breast Cancer Response Prediction from Serial DCE-MRIcited by 0
  11. 2026NV-Reason-CT: 3D Visual Language Model for CT Analysiscited by 0
  12. 2026FD-AA: A Lightweight Focal-Diffuse And Attenuation-Aware Head for Incidental Abdominal Abnormality Detection in Chest CTcited by 0
  13. 2026Scaling 3D Visual Grounding in Abdominal CTcited by 0