# Teacher-Student Architecture for Mixed Supervised Lung Tumor Segmentation

arXiv: https://arxiv.org/abs/2112.11541 · HTML: https://mlsanity.com/paper/2112.11541/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Distillation for medical imaging, one of 12 directions in Knowledge distillation.
- Builds on 21 of its 43 references that are in the map.
- First posted on arXiv 21 Dec 2021.

## Placed in

- Knowledge distillation / [Distillation for medical imaging](https://mlsanity.com/distill/distillation-for-medical-imaging/)

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations](https://mlsanity.com/paper/1707.03237/) (2017)
- [Born Again Neural Networks](https://mlsanity.com/paper/1805.04770/) (2018)
- [A large annotated medical image dataset for the development and evaluation of segmentation algorithms](https://mlsanity.com/paper/1902.09063/) (2019)
- [Towards Understanding Knowledge Distillation](https://mlsanity.com/paper/2105.13093/) (2021)
- [Dual-Teacher: Integrating Intra-domain and Inter-domain Teachers for Annotation-efficient Cardiac Segmentation](https://mlsanity.com/paper/2007.06279/) (2020)
- [EPIC: An Energy-Efficient, High-Performance GPGPU Computing Research Infrastructure](https://mlsanity.com/paper/1912.05848/) (2019)
- [Multimodal Spatial Attention Module for Targeting Multimodal PET-CT Lung Tumor Segmentation](https://mlsanity.com/paper/2007.14728/) (2020)
- [Deep Learning with Mixed Supervision for Brain Tumor Segmentation](https://mlsanity.com/paper/1812.04571/) (2018)
- [Mediastinal lymph nodes segmentation using 3D convolutional neural network ensembles and anatomical priors guiding](https://mlsanity.com/paper/2102.06515/) (2021)
- [Integrating cross-modality hallucinated MRI with CT to aid mediastinal lung tumor segmentation](https://mlsanity.com/paper/1909.04542/) (2019)
- [Cross-modality (CT-MRI) prior augmented deep learning for robust lung tumor segmentation from small MR datasets](https://mlsanity.com/paper/1901.11369/) (2019)
- [A Teacher-Student Framework for Semi-supervised Medical Image Segmentation From Mixed Supervision](https://mlsanity.com/paper/2010.12219/) (2020)
- [Lung Cancer Tumor Region Segmentation Using Recurrent 3D-DenseUNet](https://mlsanity.com/paper/1812.01951/) (2018)
- [Mixed-Supervised Dual-Network for Medical Image Segmentation](https://mlsanity.com/paper/1907.10209/) (2019)
- [Hybrid guiding: A multi-resolution refinement approach for semantic segmentation of gigapixel histopathological images](https://mlsanity.com/paper/2112.03455/) (2021)

## Abstract (arXiv)

Purpose: Automating tasks such as lung tumor localization and segmentation in radiological images can free valuable time for radiologists and other clinical personnel. Convolutional neural networks may be suited for such tasks, but require substantial amounts of labeled data to train. Obtaining labeled data is a challenge, especially in the medical domain. Methods: This paper investigates the use of a teacher-student design to utilize datasets with different types of supervision to train an automatic model performing pulmonary tumor segmentation on computed tomography images. The framework consists of two models: the student that performs end-to-end automatic tumor segmentation and the teacher that supplies the student additional pseudo-annotated data during training. Results: Using only a small proportion of semantically labeled data and a large number of bounding box annotated data, we achieved competitive performance using a teacher-student design. Models trained on larger amounts of semantic annotations did not perform better than those trained on teacher-annotated data. Conclusions: Our results demonstrate the potential of utilizing teacher-student designs to reduce the annotation load, as less supervised annotation schemes may be performed, without any real degradation in segmentation accuracy.