# MISSU: 3D Medical Image Segmentation via Self-distilling TransUNet

arXiv: https://arxiv.org/abs/2206.00902 · HTML: https://mlsanity.com/paper/2206.00902/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 54 references that are in the map.
- First posted on arXiv 2 Jun 2022.

## Builds on (most cited first, 30 of 34)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge](https://mlsanity.com/paper/1811.02629/) (2018)
- [Variational Information Distillation for Knowledge Transfer](https://mlsanity.com/paper/1904.05835/) (2019)
- [Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images](https://mlsanity.com/paper/1808.08114/) (2018)
- [TransBTS: Multimodal Brain Tumor Segmentation Using Transformer](https://mlsanity.com/paper/2103.04430/) (2021)
- [Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation](https://mlsanity.com/paper/1603.05959/) (2016)
- [Learning Lightweight Lane Detection CNNs by Self Attention Distillation](https://mlsanity.com/paper/1908.00821/) (2019)
- [Models Genesis: Generic Autodidactic Models for 3D Medical Image Analysis](https://mlsanity.com/paper/1908.06912/) (2019)
- [Multi-scale self-guided attention for medical image segmentation](https://mlsanity.com/paper/1906.02849/) (2019)
- [Unsupervised MRI Reconstruction via Zero-Shot Learned Adversarial Transformers](https://mlsanity.com/paper/2105.08059/) (2021)
- [Multi-Path Region Mining For Weakly Supervised 3D Semantic Segmentation on Point Clouds](https://mlsanity.com/paper/2003.13035/) (2020)
- [Knowledge distillation from multi-modal to mono-modal segmentation networks](https://mlsanity.com/paper/2106.09564/) (2021)
- [HyperDense-Net: A hyper-densely connected CNN for multi-modal image segmentation](https://mlsanity.com/paper/1804.02967/) (2018)
- [Multimodal Self-Supervised Learning for Medical Image Analysis](https://mlsanity.com/paper/1912.05396/) (2019)
- [Knowledge Distillation in Generations: More Tolerant Teachers Educate Better Students](https://mlsanity.com/paper/1805.05551/) (2018)
- [Efficient Medical Image Segmentation Based on Knowledge Distillation](https://mlsanity.com/paper/2108.09987/) (2021)
- [TransClaw U-Net: Claw U-Net with Transformers for Medical Image Segmentation](https://mlsanity.com/paper/2107.05188/) (2021)
- [Revisiting Rubik's Cube: Self-supervised Learning with Volume-wise Transformation for 3D Medical Image Segmentation](https://mlsanity.com/paper/2007.08826/) (2020)
- [MPRNet: Multi-Path Residual Network for Lightweight Image Super Resolution](https://mlsanity.com/paper/2011.04566/) (2020)

## Built on by (most cited first, 7 of 7)

- [A Comprehensive Survey of Convolutions in Deep Learning: Applications, Challenges, and Future Trends](https://mlsanity.com/paper/2402.15490/) (2024)
- [Volumetric medical image segmentation through dual self-distillation in U-shaped networks](https://mlsanity.com/paper/2306.03271/) (2023)
- [REHRSeg: Unleashing the Power of Self-Supervised Super-Resolution for Resource-Efficient 3D MRI Segmentation](https://mlsanity.com/paper/2410.10097/) (2024)
- [3D Brainformer: 3D Fusion Transformer for Brain Tumor Segmentation](https://mlsanity.com/paper/2304.14508/) (2023)
- [Vision Transformers increase efficiency of 3D cardiac CT multi-label segmentation](https://mlsanity.com/paper/2310.09099/) (2023)
- [MAPUNetR: A Hybrid Vision Transformer and U-Net Architecture for Efficient and Interpretable Medical Image Segmentation](https://mlsanity.com/paper/2410.22223/) (2024)
- [MSD-KMamba: Bidirectional Spatial-Aware Multi-Modal 3D Brain Segmentation via Multi-scale Self-Distilled Fusion Strategy](https://mlsanity.com/paper/2509.23677/) (2025)

## Abstract (arXiv)

U-Nets have achieved tremendous success in medical image segmentation. Nevertheless, it may suffer limitations in global (long-range) contextual interactions and edge-detail preservation. In contrast, Transformer has an excellent ability to capture long-range dependencies by leveraging the self-attention mechanism into the encoder. Although Transformer was born to model the long-range dependency on the extracted feature maps, it still suffers from extreme computational and spatial complexities in processing high-resolution 3D feature maps. This motivates us to design the efficiently Transformer-based UNet model and study the feasibility of Transformer-based network architectures for medical image segmentation tasks. To this end, we propose to self-distill a Transformer-based UNet for medical image segmentation, which simultaneously learns global semantic information and local spatial-detailed features. Meanwhile, a local multi-scale fusion block is first proposed to refine fine-grained details from the skipped connections in the encoder by the main CNN stem through self-distillation, only computed during training and removed at inference with minimal overhead. Extensive experiments on BraTS 2019 and CHAOS datasets show that our MISSU achieves the best performance over previous state-of-the-art methods. Code and models are available at urlhttps://github.com/wangn123/MISSU.git