# SegBook: A Simple Baseline and Cookbook for Volumetric Medical Image Segmentation

arXiv: https://arxiv.org/abs/2411.14525 · HTML: https://mlsanity.com/paper/2411.14525/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 69 references that are in the map.
- First posted on arXiv 21 Nov 2024.

## Builds on (most cited first, 15 of 15)

- [The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification](https://mlsanity.com/paper/2107.02314/) (2021)
- [AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation](https://mlsanity.com/paper/2206.08023/) (2022)
- [The KiTS21 Challenge: Automatic segmentation of kidneys, renal tumors, and renal cysts in corticomedullary-phase CT](https://mlsanity.com/paper/2307.01984/) (2023)
- [STU-Net: Scalable and Transferable Medical Image Segmentation Models Empowered by Large-Scale Supervised Pre-training](https://mlsanity.com/paper/2304.06716/) (2023)
- [The ASNR-MICCAI Brain Tumor Segmentation (BraTS) Challenge 2023: Intracranial Meningioma](https://mlsanity.com/paper/2305.07642/) (2023)
- [The Brain Tumor Segmentation (BraTS) Challenge 2023: Glioma Segmentation in Sub-Saharan Africa Patient Population (BraTS-Africa)](https://mlsanity.com/paper/2305.19369/) (2023)
- [Unleashing the Strengths of Unlabeled Data in Pan-cancer Abdominal Organ Quantification: the FLARE22 Challenge](https://mlsanity.com/paper/2308.05862/) (2023)
- [The Brain Tumor Segmentation (BraTS) Challenge 2023: Focus on Pediatrics (CBTN-CONNECT-DIPGR-ASNR-MICCAI BraTS-PEDs)](https://mlsanity.com/paper/2305.17033/) (2023)
- [CTSpine1K: A Large-Scale Dataset for Spinal Vertebrae Segmentation in Computed Tomography](https://mlsanity.com/paper/2105.14711/) (2021)
- [The Brain Tumor Segmentation (BraTS-METS) Challenge 2023: Brain Metastasis Segmentation on Pre-treatment MRI](https://mlsanity.com/paper/2306.00838/) (2023)
- [SegRap2023: A Benchmark of Organs-at-Risk and Gross Tumor Volume Segmentation for Radiotherapy Planning of Nasopharyngeal Carcinoma](https://mlsanity.com/paper/2312.09576/) (2023)
- [LNDb: A Lung Nodule Database on Computed Tomography](https://mlsanity.com/paper/1911.08434/) (2019)
- [Efficient automatic segmentation for multi-level pulmonary arteries: The PARSE challenge](https://mlsanity.com/paper/2304.03708/) (2023)
- [The state-of-the-art 3D anisotropic intracranial hemorrhage segmentation on non-contrast head CT: The INSTANCE challenge](https://mlsanity.com/paper/2301.03281/) (2023)
- [The Extreme Cardiac MRI Analysis Challenge under Respiratory Motion (CMRxMotion)](https://mlsanity.com/paper/2210.06385/) (2022)

## Abstract (arXiv)

Computed Tomography (CT) is one of the most popular modalities for medical imaging. By far, CT images have contributed to the largest publicly available datasets for volumetric medical segmentation tasks, covering full-body anatomical structures. Large amounts of full-body CT images provide the opportunity to pre-train powerful models, e.g., STU-Net pre-trained in a supervised fashion, to segment numerous anatomical structures. However, it remains unclear in which conditions these pre-trained models can be transferred to various downstream medical segmentation tasks, particularly segmenting the other modalities and diverse targets. To address this problem, a large-scale benchmark for comprehensive evaluation is crucial for finding these conditions. Thus, we collected 87 public datasets varying in modality, target, and sample size to evaluate the transfer ability of full-body CT pre-trained models. We then employed a representative model, STU-Net with multiple model scales, to conduct transfer learning across modalities and targets. Our experimental results show that (1) there may be a bottleneck effect concerning the dataset size in fine-tuning, with more improvement on both small- and large-scale datasets than medium-size ones. (2) Models pre-trained on full-body CT demonstrate effective modality transfer, adapting well to other modalities such as MRI. (3) Pre-training on the full-body CT not only supports strong performance in structure detection but also shows efficacy in lesion detection, showcasing adaptability across target tasks. We hope that this large-scale open evaluation of transfer learning can direct future research in volumetric medical image segmentation.