Expectation-Maximization as the Engine of Scalable Medical Intelligence
Not on the map. Why?
Abstract (arXiv)
Large, high-quality, annotated datasets are the foundation of medical AI research, but constructing even a small, moderate-quality, annotated dataset can take years of effort from multidisciplinary teams. Although active learning can prioritize what to annotate, scaling up still requires extensive manual efforts to revise the noisy annotations. We formulate this as a missing-data problem and develop ScaleMAI, a framework that unifies data annotation and model development co-evolution through an Expectation-Maximization (EM) process. In this iterative process, the AI model automatically identifies and corrects the mistakes in annotations (Expectation), while the refined annotated data retrain the model to improve accuracy (Maximization). In addition to the classical EM algorithm, ScaleMAI brings human experts into the loop to review annotations that cannot be adequately addressed by either Expectation or Maximization step (<5%). As a result, ScaleMAI progressively creates an annotated dataset of 47,315 CT scans (4.8x larger than the largest public dataset, PanTS) including 4,163,720 per-voxel annotations for benign/malignant tumors and 88 anatomical structures. ScaleMAI iteratively trains a model that exceeds human expert performance in tumor diagnosis (+7%), and outperforms models developed from smaller, moderate-quality datasets, with statistically significant gains in tumor detection (+10%) and segmentation (+14%) on two prestigious benchmarks.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 44 of its 78 references that are in the map. First posted on arXiv 6 Jan 2025.
Builds on
44 of 78 references.
References · 44
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2023GPT-4 Technical Reportcited by 18,132
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2023Segment Anythingcited by 8,158
- 2023Visual Instruction Tuningcited by 7,792
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
- 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
- 2023DataComp: In search of the next generation of multimodal datasetscited by 635
32 more
- 2021The Medical Segmentation Decathloncited by 620
- 2022TotalSegmentator: robust segmentation of 104 anatomical structures in CT imagescited by 597
- 2019The Liver Tumor Segmentation Benchmark (LiTS)cited by 504
- 2022MONAI: An open-source framework for deep learning in healthcarecited by 438
- 2021Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysiscited by 418
- 2021Domain Adaptation for Medical Image Analysis: A Surveycited by 351
- 2022AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentationcited by 347
- 2024nnU-Net Revisited: A Call for Rigorous Validation in 3D Medical Image Segmentationcited by 235
- 2023CLIP-Driven Universal Model for Organ Segmentation and Tumor Detectioncited by 224
- 2016Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learningcited by 209
- 2019The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomescited by 168
- 2023MedNeXt: Transformer-driven Scaling of ConvNets for Medical Image Segmentationcited by 161
- 2019Models Genesis: Generic Autodidactic Models for 3D Medical Image Analysiscited by 159
- 2015DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentationcited by 158
- 2023STU-Net: Scalable and Transferable Medical Image Segmentation Models Empowered by Large-Scale Supervised Pre-trainingcited by 96
- 2024AbdomenAtlas: A Large-Scale, Detailed-Annotated, & Multi-Center Dataset for Efficient Transfer Learning and Open Algorithmic Benchmarkingcited by 64
- 2019Methods and open-source toolkit for analyzing and visualizing challenge resultscited by 60
- 2023Label-Free Liver Tumor Segmentationcited by 56
- 2019C2FNAS: Coarse-to-Fine Neural Architecture Search for 3D Medical Image Segmentationcited by 49
- 2021DiNTS: Differentiable Neural Network Topology Search for 3D Medical Image Segmentationcited by 47
- 2023UniSeg: A Prompt-driven Universal Segmentation Model as well as A Strong Representation Learnercited by 46
- 2017Reverse Classification Accuracy: Predicting Segmentation Performance in the Absence of Ground Truthcited by 35
- 2023MedShapeNet -- A Large-Scale Dataset of 3D Medical Shapes for Computer Visioncited by 31
- 2024Universal and Extensible Language-Vision Models for Organ Segmentation and Tumor Detection from Abdominal Computed Tomographycited by 31
- 2024Touchstone Benchmark: Are We on the Right Way for Evaluating AI Algorithms for Medical Segmentation?cited by 31
- 2022Making Your First Choice: To Address Cold Start Problem in Vision Active Learningcited by 25
- 2024Automatic Organ and Pan-cancer Segmentation in Abdomen CT: the FLARE 2023 Challengecited by 20
- 2023Boosting Dermatoscopic Lesion Segmentation via Diffusion Models with Visual and Textual Promptscited by 17
- 2022Synthetic Tumors Make AI Segment Tumors Bettercited by 14
- 2024The RSNA Abdominal Traumatic Injury CT (RATIC) Datasetcited by 12
- 2023Synthetic Data as Validationcited by 9
- 2024Embracing Massive Medical Datacited by 9
Built on by
Citing papers per year: 2025: 4; 2026: 1.
5 papers.
- 2025Scaling Tumor Segmentation: Best Lessons from Real and Synthetic Datacited by 13
- 2025Learning Segmentation from Radiology Reportscited by 11
- 2025Medal S: Spatio-Textual Prompt Model for Medical Segmentationcited by 1
- 2025ShapeKitcited by 0
- 2026Merlin Plus: A Large-Scale, Multi-Cancer, Image-Mask-Report Datasetcited by 0
- 2026Merlin Plus: A Large-Scale, Multi-Cancer, Image-Mask-Report Dataset29 Sep 2026
- 2025Medal S: Spatio-Textual Prompt Model for Medical Segmentation17 Nov 2025
- 2025Scaling Tumor Segmentation: Best Lessons from Real and Synthetic Data16 Oct 2025
- 2025Learning Segmentation from Radiology Reports8 Jul 2025
- 2025ShapeKit30 Jun 2025