# Designing UNICORN: a Unified Benchmark for Imaging in Computational Pathology, Radiology, and Natural Language

arXiv: https://arxiv.org/abs/2603.02790 · HTML: https://mlsanity.com/paper/2603.02790/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 57 references that are in the map.
- First posted on arXiv 3 Mar 2026.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Phi-4 Technical Report](https://mlsanity.com/paper/2412.08905/) (2024)
- [The Medical Segmentation Decathlon](https://mlsanity.com/paper/2106.05735/) (2021)
- [MedCLIP: Contrastive Learning from Unpaired Medical Images and Text](https://mlsanity.com/paper/2210.10163/) (2022)
- [HealthBench: Evaluating Large Language Models Towards Improved Human Health](https://mlsanity.com/paper/2505.08775/) (2025)
- [CHAOS Challenge -- Combined (CT-MR) Healthy Abdominal Organ Segmentation](https://mlsanity.com/paper/2001.06535/) (2020)
- [Towards Generalist Biomedical AI](https://mlsanity.com/paper/2307.14334/) (2023)
- [Capabilities of Gemini Models in Medicine](https://mlsanity.com/paper/2404.18416/) (2024)
- [Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing](https://mlsanity.com/paper/2301.04558/) (2023)
- [Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge](https://mlsanity.com/paper/1612.08012/) (2016)
- [Quilt-1M: One Million Image-Text Pairs for Histopathology](https://mlsanity.com/paper/2306.11207/) (2023)
- [Learn2Reg: comprehensive multi-task medical image registration challenge, dataset and evaluation in the era of deep learning](https://mlsanity.com/paper/2112.04489/) (2021)
- [PRISM: A Multi-Modal Generative Foundation Model for Slide-Level Histopathology](https://mlsanity.com/paper/2405.10254/) (2024)
- [USFM: A Universal Ultrasound Foundation Model Generalized to Tasks and Organs towards Label Efficient Image Analysis](https://mlsanity.com/paper/2401.00153/) (2023)
- [A Clinical Benchmark of Public Self-Supervised Pathology Foundation Models](https://mlsanity.com/paper/2407.06508/) (2024)
- [MedHELM: Holistic Evaluation of Large Language Models for Medical Tasks](https://mlsanity.com/paper/2505.23802/) (2025)
- [ELIXR: Towards a general purpose X-ray artificial intelligence system through alignment of large language models and radiology vision encoders](https://mlsanity.com/paper/2308.01317/) (2023)
- [FairMedFM: Fairness Benchmarking for Medical Imaging Foundation Models](https://mlsanity.com/paper/2407.00983/) (2024)
- [Vision Foundation Models for Computed Tomography](https://mlsanity.com/paper/2501.09001/) (2025)
- [MedFMC: A Real-world Dataset and Benchmark For Foundation Model Adaptation in Medical Image Classification](https://mlsanity.com/paper/2306.09579/) (2023)
- [WsiCaption: Multiple Instance Generation of Pathology Reports for Gigapixel Whole-Slide Images](https://mlsanity.com/paper/2311.16480/) (2023)
- [Lumbar spine segmentation in MR images: a dataset and a public benchmark](https://mlsanity.com/paper/2306.12217/) (2023)
- [The ULS23 Challenge: a Baseline Model and Benchmark Dataset for 3D Universal Lesion Segmentation in Computed Tomography](https://mlsanity.com/paper/2406.05231/) (2024)
- [MRSegmentator: Multi-Modality Segmentation of 40 Classes in MRI and CT](https://mlsanity.com/paper/2405.06463/) (2024)

## Built on by (most cited first, 3 of 3)

- [DALPHIN: Benchmarking Digital Pathology AI Copilots Against Pathologists on an Open Multicentric Dataset](https://mlsanity.com/paper/2605.03544/) (2026)
- [Benchmarking Foundation Models for Renal Lesion Stratification in CT](https://mlsanity.com/paper/2605.07749/) (2026)
- [CoM3eT: A foundation model for medical image analysis through federated, multidimensional context integration](https://mlsanity.com/paper/2608.16268/) (2026)

## Abstract (arXiv)

Foundation models are changing the way we develop medical artificial intelligence. By learning broadly generalizable features across diverse data modalities, a single model can be rapidly adapted to address multiple modalities and tasks with minimal supervision. This potential comes with the urgent need to reliably benchmark, understand and compare the performance and clinical impact of foundation models across data modalities and clinical tasks. We introduce UNICORN, a fundamentally new benchmarking concept for medical foundation models. UNICORN brings four main contributions to medical artificial intelligence. First, a framework that enables a one-to-many benchmarking approach, where a single foundation model is tested across multiple tasks and data modalities. Here, we populate it with 20 tasks across radiology, pathology, and clinical text, covering classification, detection, segmentation, regression, and vision-language generation. Second, a publicly available evaluation platform that implements, for the first time, a two-step approach to run foundation models for data encoding followed by custom task-specific adaptation via few-shot learning and linear probing mechanisms. Third, we create a meta-model that combines state-of-the-art foundation models in pathology, radiology and language with novel task-specific adapters that address all UNICORN tasks, which we refer to as Unicorn Model-0 (UM-0). Finally, we design a novel UNICORN score to benchmark and compare model performance across all tasks. We present the results of UM-0 using sequestered test data from over 2,400 patients, 3,700 vision cases, and 2,400 clinical reports from 17 institutions across eight countries, spanning eight anatomical regions and four imaging modalities. Data, baselines, and evaluation platform are publicly accessible at unicorn.grand-challenge.org.