# Alternate Diverse Teaching for Semi-supervised Medical Image Segmentation

arXiv: https://arxiv.org/abs/2311.17325 · HTML: https://mlsanity.com/paper/2311.17325/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Distillation for medical imaging, one of 12 directions in Knowledge distillation.
- Builds on 36 of its 49 references that are in the map.
- First posted on arXiv 29 Nov 2023.

## Placed in

- Knowledge distillation / [Distillation for medical imaging](https://mlsanity.com/distill/distillation-for-medical-imaging/)

## Builds on (most cited first, 30 of 36)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation](https://mlsanity.com/paper/2012.07177/) (2020)
- [Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning](https://mlsanity.com/paper/1908.02983/) (2019)
- [Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision](https://mlsanity.com/paper/2106.01226/) (2021)
- [Uncertainty-aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentation](https://mlsanity.com/paper/1907.07034/) (2019)
- [ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring](https://mlsanity.com/paper/1911.09785/) (2019)
- [Semi-supervised Medical Image Segmentation through Dual-task Consistency](https://mlsanity.com/paper/2009.04448/) (2020)
- [UNeXt: MLP-based Rapid Medical Image Segmentation Network](https://mlsanity.com/paper/2203.04967/) (2022)
- [ST++: Make Self-training Work Better for Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/2106.05095/) (2021)
- [Revisiting Weak-to-Strong Consistency in Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2208.09910/) (2022)
- [DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation](https://mlsanity.com/paper/1506.06448/) (2015)
- [Shape-aware Semi-supervised 3D Semantic Segmentation for Medical Images](https://mlsanity.com/paper/2007.10732/) (2020)
- [Bidirectional Copy-Paste for Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2305.00673/) (2023)
- [Perturbed and Strict Mean Teachers for Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/2111.12903/) (2021)
- [Mutual Consistency Learning for Semi-supervised Medical Image Segmentation](https://mlsanity.com/paper/2109.09960/) (2021)
- [Semi-supervised Left Atrium Segmentation with Mutual Consistency Training](https://mlsanity.com/paper/2103.02911/) (2021)
- [Efficient Semi-Supervised Gross Target Volume of Nasopharyngeal Carcinoma Segmentation via Uncertainty Rectified Pyramid Consistency](https://mlsanity.com/paper/2012.07042/) (2020)
- [Semi-Supervised Medical Image Segmentation via Cross Teaching between CNN and Transformer](https://mlsanity.com/paper/2112.04894/) (2021)
- [Exploring Smoothness and Class-Separation for Semi-supervised Medical Image Segmentation](https://mlsanity.com/paper/2203.01324/) (2022)
- [Bootstrapping Semantic Segmentation with Regional Contrast](https://mlsanity.com/paper/2104.04465/) (2021)
- [A Simple Baseline for Semi-supervised Semantic Segmentation with Strong Data Augmentation](https://mlsanity.com/paper/2104.07256/) (2021)
- [Augmentation Matters: A Simple-yet-Effective Approach to Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/2212.04976/) (2022)
- [Semi-supervised Semantic Segmentation with Error Localization Network](https://mlsanity.com/paper/2204.02078/) (2022)

## Built on by (most cited first, 22 of 22)

- [UniMatch V2: Pushing the Limit of Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2410.10777/) (2024)
- [MetaSSL: A General Heterogeneous Loss for Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2509.01144/) (2025)
- [Feedback-Driven Pseudo-Label Reliability Assessment: Redefining Thresholding for Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2505.07691/) (2025)
- [Dual Cross-image Semantic Consistency with Self-aware Pseudo Labeling for Semi-supervised Medical Image Segmentation](https://mlsanity.com/paper/2507.21440/) (2025)
- [HDC: Hierarchical Distillation for Multi-level Noisy Consistency in Semi-Supervised Fetal Ultrasound Segmentation](https://mlsanity.com/paper/2504.09876/) (2025)
- [SSS: Semi-Supervised SAM-2 with Efficient Prompting for Medical Imaging Segmentation](https://mlsanity.com/paper/2506.08949/) (2025)
- [DiN: Diffusion Model for Robust Medical VQA with Semantic Noisy Labels](https://mlsanity.com/paper/2503.18536/) (2025)
- [Adaptive Knowledge Transferring with Switching Dual-Student Framework for Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2510.24366/) (2025)
- [Leveraging Labelled Data Knowledge: A Cooperative Rectification Learning Network for Semi-supervised 3D Medical Image Segmentation](https://mlsanity.com/paper/2502.11456/) (2025)
- [Segmenting Visuals With Querying Words: Language Anchors For Semi-Supervised Image Segmentation](https://mlsanity.com/paper/2506.13925/) (2025)
- [ARTEMIS: Agent-guided Reliability-aware Temporal Mask Evolution for Imperfectly Supervised Video Polyp Segmentation](https://mlsanity.com/paper/2606.20161/) (2026)
- [Dual-Teacher Ensemble Models with Double-Copy-Paste for 3D Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2410.11509/) (2024)
- [Dual Teacher-Student Learning for Semi-supervised Medical Image Segmentation](https://mlsanity.com/paper/2505.11018/) (2025)
- [P3Net: Progressive and Periodic Perturbation for Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2505.15861/) (2025)
- [Style-Aware Blending and Prototype-Based Cross-Contrast Consistency for Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2507.20729/) (2025)
- [Semi-Supervised Dual-Threshold Contrastive Learning for Ultrasound Image Classification and Segmentation](https://mlsanity.com/paper/2508.02265/) (2025)
- [M3HL: Mutual Mask Mix with High-Low Level Feature Consistency for Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2508.03752/) (2025)
- [Semi-Supervised 3D Medical Segmentation from 2D Natural Images Pretrained Model](https://mlsanity.com/paper/2509.15167/) (2025)
- [Intra-Class Subdivision for Pixel Contrastive Learning: Application to Semi-supervised Cardiac Image Segmentation](https://mlsanity.com/paper/2602.00174/) (2026)
- [SemiGDA: Generative Dual-distribution Alignment for Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2604.23274/) (2026)
- [Confidence-Aware Teacher-Student Distillation for 3D Medical Segmentation](https://mlsanity.com/paper/2609.23815/) (2026)
- [Human-Inspired Asymmetric Collaboration for Dual-Student Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2610.18133/) (2026)

## Abstract (arXiv)

Semi-supervised medical image segmentation studies have shown promise in training models with limited labeled data. However, current dominant teacher-student based approaches can suffer from the confirmation bias. To address this challenge, we propose AD-MT, an alternate diverse teaching approach in a teacher-student framework. It involves a single student model and two non-trainable teacher models that are momentum-updated periodically and randomly in an alternate fashion. To mitigate the confirmation bias from the diverse supervision, the core of AD-MT lies in two proposed modules: the Random Periodic Alternate (RPA) Updating Module and the Conflict-Combating Module (CCM). The RPA schedules the alternating diverse updating process with complementary data batches, distinct data augmentation, and random switching periods to encourage diverse reasoning from different teaching perspectives. The CCM employs an entropy-based ensembling strategy to encourage the model to learn from both the consistent and conflicting predictions between the teachers. Experimental results demonstrate the effectiveness and superiority of our AD-MT on the 2D and 3D medical segmentation benchmarks across various semi-supervised settings.