# Enhancing Single-Slice Segmentation with 3D-to-2D Unpaired Scan Distillation

arXiv: https://arxiv.org/abs/2406.12254 · HTML: https://mlsanity.com/paper/2406.12254/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 24 references that are in the map.
- First posted on arXiv 18 Jun 2024.

## Builds on (most cited first, 16 of 16)

- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis](https://mlsanity.com/paper/2111.14791/) (2021)
- [Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2101.10979/) (2021)
- [Knowledge Distillation from A Stronger Teacher](https://mlsanity.com/paper/2205.10536/) (2022)
- [Unpaired Multi-modal Segmentation via Knowledge Distillation](https://mlsanity.com/paper/2001.03111/) (2020)
- [Pri3D: Can 3D Priors Help 2D Representation Learning?](https://mlsanity.com/paper/2104.11225/) (2021)
- [3D-to-2D Distillation for Indoor Scene Parsing](https://mlsanity.com/paper/2104.02243/) (2021)
- [Learnable Cross-modal Knowledge Distillation for Multi-modal Learning with Missing Modality](https://mlsanity.com/paper/2310.01035/) (2023)
- [Efficient Medical Image Segmentation Based on Knowledge Distillation](https://mlsanity.com/paper/2108.09987/) (2021)
- [Prototype Knowledge Distillation for Medical Segmentation with Missing Modality](https://mlsanity.com/paper/2303.09830/) (2023)
- [COSST: Multi-organ Segmentation with Partially Labeled Datasets Using Comprehensive Supervisions and Self-training](https://mlsanity.com/paper/2304.14030/) (2023)
- [Scaling Up 3D Kernels with Bayesian Frequency Re-parameterization for Medical Image Segmentation](https://mlsanity.com/paper/2303.05785/) (2023)
- [FoPro-KD: Fourier Prompted Effective Knowledge Distillation for Long-Tailed Medical Image Recognition](https://mlsanity.com/paper/2305.17421/) (2023)
- [Reducing Positional Variance in Cross-sectional Abdominal CT Slices with Deep Conditional Generative Models](https://mlsanity.com/paper/2209.14467/) (2022)
- [Longitudinal Variability Analysis on Low-dose Abdominal CT with Deep Learning-based Segmentation](https://mlsanity.com/paper/2209.14217/) (2022)

## Built on by (most cited first, 2 of 2)

- [CT-Agent: A Multimodal-LLM Agent for 3D CT Radiology Question Answering](https://mlsanity.com/paper/2505.16229/) (2025)
- [MOSAIC: A Multi-View 2.5D Organ Slice Selector with Cross-Attentional Reasoning for Anatomically-Aware CT Localization in Medical Organ Segmentation](https://mlsanity.com/paper/2505.10672/) (2025)

## Abstract (arXiv)

2D single-slice abdominal computed tomography (CT) enables the assessment of body habitus and organ health with low radiation exposure. However, single-slice data necessitates the use of 2D networks for segmentation, but these networks often struggle to capture contextual information effectively. Consequently, even when trained on identical datasets, 3D networks typically achieve superior segmentation results. In this work, we propose a novel 3D-to-2D distillation framework, leveraging pre-trained 3D models to enhance 2D single-slice segmentation. Specifically, we extract the prediction distribution centroid from the 3D representations, to guide the 2D student by learning intra- and inter-class correlation. Unlike traditional knowledge distillation methods that require the same data input, our approach employs unpaired 3D CT scans with any contrast to guide the 2D student model. Experiments conducted on 707 subjects from the single-slice Baltimore Longitudinal Study of Aging (BLSA) dataset demonstrate that state-of-the-art 2D multi-organ segmentation methods can benefit from the 3D teacher model, achieving enhanced performance in single-slice multi-organ segmentation. Notably, our approach demonstrates considerable efficacy in low-data regimes, outperforming the model trained with all available training subjects even when utilizing only 200 training subjects. Thus, this work underscores the potential to alleviate manual annotation burdens.