# Models Genesis

arXiv: https://arxiv.org/abs/2004.07882 · HTML: https://mlsanity.com/paper/2004.07882/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Apr 2020.

## Built on by (most cited first, 26 of 26)

- [Knowledge-enhanced Visual-Language Pre-training on Chest Radiology Images](https://mlsanity.com/paper/2302.14042/) (2023)
- [Denoising Diffusion Probabilistic Models for Generation of Realistic Fully-Annotated Microscopy Image Data Sets](https://mlsanity.com/paper/2301.10227/) (2023)
- [Revisiting 2D Foundation Models for Scalable 3D Medical Image Classification](https://mlsanity.com/paper/2512.12887/) (2025)
- [LNQ 2023 challenge: Benchmark of weakly-supervised techniques for mediastinal lymph node quantification](https://mlsanity.com/paper/2408.10069/) (2024)
- [Structure-Guided Self-Supervised Matching for One-Shot Medical Landmark Detection](https://mlsanity.com/paper/2203.01687/) (2022)
- [Large-scale Multi-sequence Pretraining for Generalizable MRI Analysis in Versatile Clinical Applications](https://mlsanity.com/paper/2508.07165/) (2025)
- [Deep Cervix Model Development from Heterogeneous and Partially Labeled Image Datasets](https://mlsanity.com/paper/2201.07013/) (2022)
- [Weakly Supervised Lymph Nodes Segmentation Based on Partial Instance Annotations with Pre-trained Dual-branch Network and Pseudo Label Learning](https://mlsanity.com/paper/2408.09411/) (2024)
- [Topology-Driven Transferability Estimation of Medical Foundation Models for Segmentation](https://mlsanity.com/paper/2602.23916/) (2026)
- [GloResNet: A lightweight 3D CNN with global topological features for preterm brain injury prediction](https://mlsanity.com/paper/2606.02498/) (2026)
- [Topology-Driven Transferability Estimation for 3D Medical Vision Foundation Models](https://mlsanity.com/paper/2607.04199/) (2026)
- [When Do Cheap Probes Predict Expensive Training? Probing 3D-CT Encoders for Text Generation](https://mlsanity.com/paper/2607.22771/) (2026)
- [Rad-JEPA 3D: Radiology Joint-Embedding Predictive Model for 3D Computed Tomography](https://mlsanity.com/paper/2607.26196/) (2026)
- [Anatomy Contextualized Adaptation of CT Foundation Models](https://mlsanity.com/paper/2607.27154/) (2026)
- [Cross-Modal Fine-Tuning of 3D Convolutional Foundation Models for ADHD Classification with Low-Rank Adaptation](https://mlsanity.com/paper/2511.06163/) (2025)
- [AFFMAE: Scalable Vision Pre-Training for High-Resolution Microscopy Segmentation on Desktop Hardware](https://mlsanity.com/paper/2602.16249/) (2026)
- [Knowledge Transfer Scaling Laws for 3D Medical Imaging](https://mlsanity.com/paper/2605.06859/) (2026)
- [Benchmarking transferability of SSL pretraining to same and different modality segmentation tasks](https://mlsanity.com/paper/2605.18491/) (2026)
- [Multi-Granularity 3D Kidney Lesion Characterization from CT Volumes](https://mlsanity.com/paper/2606.04365/) (2026)
- [Evaluating self-supervised echocardiographic representations across downstream extraction strategies for left-ventricular segmentation and ejection fraction estimation](https://mlsanity.com/paper/2606.22943/) (2026)
- [OrganLens: Organ-Specific Representation Learning for CT Foundation Models](https://mlsanity.com/paper/2607.25164/) (2026)
- [Lesion Detection in CT with Frozen Self-Distilled Features: SALT, a Spatially Adaptive Label-Guided Temperature](https://mlsanity.com/paper/2608.05100/) (2026)
- [Report Supervision](https://mlsanity.com/paper/2608.27668/) (2026)
- [MedSegBenchmarker: A Raw-Count-First Framework for Controlled 2D Medical Image Segmentation Benchmarks](https://mlsanity.com/paper/2608.29677/) (2026)
- [ConPro: Contrast Projection Pretraining for Label-Efficient Vessel Segmentation in DSA Sequences](https://mlsanity.com/paper/2609.30043/) (2026)
- [RT-Super: Learning Tumor Segmentation from Longitudinal Images and Reports](https://mlsanity.com/paper/2609.35637/) (2026)

## Abstract (arXiv)

Transfer learning from natural images to medical images has been established as one of the most practical paradigms in deep learning for medical image analysis. To fit this paradigm, however, 3D imaging tasks in the most prominent imaging modalities (e.g., CT and MRI) have to be reformulated and solved in 2D, losing rich 3D anatomical information, thereby inevitably compromising its performance. To overcome this limitation, we have built a set of models, called Generic Autodidactic Models, nicknamed Models Genesis, because they are created ex nihilo (with no manual labeling), self-taught (learnt by self-supervision), and generic (served as source models for generating application-specific target models). Our extensive experiments demonstrate that our Models Genesis significantly outperform learning from scratch and existing pre-trained 3D models in all five target 3D applications covering both segmentation and classification. More importantly, learning a model from scratch simply in 3D may not necessarily yield performance better than transfer learning from ImageNet in 2D, but our Models Genesis consistently top any 2D/2.5D approaches including fine-tuning the models pre-trained from ImageNet as well as fine-tuning the 2D versions of our Models Genesis, confirming the importance of 3D anatomical information and significance of Models Genesis for 3D medical imaging. This performance is attributed to our unified self-supervised learning framework, built on a simple yet powerful observation: the sophisticated and recurrent anatomy in medical images can serve as strong yet free supervision signals for deep models to learn common anatomical representation automatically via self-supervision. As open science, all codes and pre-trained Models Genesis are available at https://github.com/MrGiovanni/ModelsGenesis.