# ACT-Net: Asymmetric Co-Teacher Network for Semi-supervised Memory-efficient Medical Image Segmentation

arXiv: https://arxiv.org/abs/2207.01900 · HTML: https://mlsanity.com/paper/2207.01900/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Distillation for medical imaging, one of 12 directions in Knowledge distillation.
- Builds on 18 of its 24 references that are in the map.
- First posted on arXiv 5 Jul 2022.

## Placed in

- Knowledge distillation / [Distillation for medical imaging](https://mlsanity.com/distill/distillation-for-medical-imaging/)

## Builds on (most cited first, 18 of 18)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results](https://mlsanity.com/paper/1703.01780/) (2017)
- [On the Efficacy of Knowledge Distillation](https://mlsanity.com/paper/1910.01348/) (2019)
- [Uncertainty-aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentation](https://mlsanity.com/paper/1907.07034/) (2019)
- [A Comprehensive Overhaul of Feature Distillation](https://mlsanity.com/paper/1904.01866/) (2019)
- [Correlation Congruence for Knowledge Distillation](https://mlsanity.com/paper/1904.01802/) (2019)
- [Paraphrasing Complex Network: Network Compression via Factor Transfer](https://mlsanity.com/paper/1802.04977/) (2018)
- [Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation](https://mlsanity.com/paper/1908.10454/) (2019)
- [Semi-supervised Medical Image Segmentation through Dual-task Consistency](https://mlsanity.com/paper/2009.04448/) (2020)
- [Suggestive Annotation: A Deep Active Learning Framework for Biomedical Image Segmentation](https://mlsanity.com/paper/1706.04737/) (2017)
- [Semi-supervised Contrastive Learning for Label-efficient Medical Image Segmentation](https://mlsanity.com/paper/2109.07407/) (2021)
- [DSAL: Deeply Supervised Active Learning from Strong and Weak Labelers for Biomedical Image Segmentation](https://mlsanity.com/paper/2101.09057/) (2021)
- [Hierarchical Consistency Regularized Mean Teacher for Semi-supervised 3D Left Atrium Segmentation](https://mlsanity.com/paper/2105.10369/) (2021)
- [BiRA-Net: Bilinear Attention Net for Diabetic Retinopathy Grading](https://mlsanity.com/paper/1905.06312/) (2019)
- [Low complexity convolutional neural network for vessel segmentation in portable retinal diagnostic devices](https://mlsanity.com/paper/1802.07804/) (2018)

## Built on by (most cited first, 4 of 4)

- [Machine Learning for Synthetic Data Generation: A Review](https://mlsanity.com/paper/2302.04062/) (2023)
- [Improving Differentiable Architecture Search via Self-Distillation](https://mlsanity.com/paper/2302.05629/) (2023)
- [Semi-Supervised Crowd Counting with Contextual Modeling: Facilitating Holistic Understanding of Crowd Scenes](https://mlsanity.com/paper/2310.10352/) (2023)
- [Contour-weighted loss for class-imbalanced image segmentation](https://mlsanity.com/paper/2407.06176/) (2024)

## Abstract (arXiv)

While deep models have shown promising performance in medical image segmentation, they heavily rely on a large amount of well-annotated data, which is difficult to access, especially in clinical practice. On the other hand, high-accuracy deep models usually come in large model sizes, limiting their employment in real scenarios. In this work, we propose a novel asymmetric co-teacher framework, ACT-Net, to alleviate the burden on both expensive annotations and computational costs for semi-supervised knowledge distillation. We advance teacher-student learning with a co-teacher network to facilitate asymmetric knowledge distillation from large models to small ones by alternating student and teacher roles, obtaining tiny but accurate models for clinical employment. To verify the effectiveness of our ACT-Net, we employ the ACDC dataset for cardiac substructure segmentation in our experiments. Extensive experimental results demonstrate that ACT-Net outperforms other knowledge distillation methods and achieves lossless segmentation performance with 250x fewer parameters.