mlsanityData to 14 Oct 2026

Dual-Teacher Ensemble Models with Double-Copy-Paste for 3D Semi-Supervised Medical Image Segmentation

arXiv 2410.11509 ↗First posted 15 Oct 2024cs.CVcited by 0

On the map: Knowledge distillation → Distillation for medical imaging

Abstract (arXiv)

Semi-supervised learning (SSL) techniques address the high labeling costs in 3D medical image segmentation, with the teacher-student model being a common approach. However, using an exponential moving average (EMA) in single-teacher models may cause coupling issues, where the weights of the student and teacher models become similar, limiting the teacher's ability to provide additional knowledge for the student. Dual-teacher models were introduced to address this problem but often neglected the importance of maintaining teacher model diversity, leading to coupling issues among teachers. To address the coupling issue, we incorporate a double-copy-paste (DCP) technique to enhance the diversity among the teachers. Additionally, we introduce the Staged Selective Ensemble (SSE) module, which selects different ensemble methods based on the characteristics of the samples and enables more accurate segmentation of label boundaries, thereby improving the quality of pseudo-labels. Experimental results demonstrate the effectiveness of our proposed method in 3D medical image segmentation tasks. Here is the code link: https://github.com/Fazhan-cs/DCP.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Distillation for medical imaging, one of 12 directions in Knowledge distillation. Builds on 32 of its 39 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

32 of 39 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Alternate Diverse Teaching for Semi-supervised Medical Image Segmentation29 Nov

Other references · 31

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  3. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  4. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,292
  5. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  6. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,476
  7. 2020Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentationcited by 596
  8. 2019Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learningcited by 510
  9. 2021Semi-Supervised Semantic Segmentation with Cross Pseudo Supervisioncited by 463
  10. 2019Uncertainty-aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentationcited by 379
  11. 2020Semi-supervised Medical Image Segmentation through Dual-task Consistencycited by 216
  12. 2015DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentationcited by 159
19 more
  1. 2020Shape-aware Semi-supervised 3D Semantic Segmentation for Medical Imagescited by 152
  2. 2023Bidirectional Copy-Paste for Semi-Supervised Medical Image Segmentationcited by 116
  3. 2021Perturbed and Strict Mean Teachers for Semi-supervised Semantic Segmentationcited by 113
  4. 2021Mutual Consistency Learning for Semi-supervised Medical Image Segmentationcited by 108
  5. 2019Dual Student: Breaking the Limits of the Teacher in Semi-supervised Learningcited by 104
  6. 2020Efficient Semi-Supervised Gross Target Volume of Nasopharyngeal Carcinoma Segmentation via Uncertainty Rectified Pyramid Consistencycited by 102
  7. 2021Semi-Supervised Medical Image Segmentation via Cross Teaching between CNN and Transformercited by 100
  8. 2022Exploring Smoothness and Class-Separation for Semi-supervised Medical Image Segmentationcited by 93
  9. 2022Learning with Limited Annotations: A Survey on Deep Semi-Supervised Learning for Medical Image Segmentationcited by 76
  10. 2021Inconsistency-aware Uncertainty Estimation for Semi-supervised Medical Image Segmentationcited by 61
  11. 2022UCC: Uncertainty guided Cross-head Co-training for Semi-Supervised Semantic Segmentationcited by 45
  12. 2019Simulation of hyperelastic materials in real-time using Deep Learningcited by 35
  13. 2023Towards Generic Semi-Supervised Framework for Volumetric Medical Image Segmentationcited by 35
  14. 2023Correlation-Aware Mutual Learning for Semi-supervised Medical Image Segmentationcited by 30
  15. 2022Bootstrapping Semi-supervised Medical Image Segmentation with Anatomical-aware Contrastive Distillationcited by 27
  16. 2022CP2: Copy-Paste Contrastive Pretraining for Semantic Segmentationcited by 26
  17. 2021All-Around Real Label Supervision: Cyclic Prototype Consistency Learning for Semi-supervised Medical Image Segmentationcited by 25
  18. 2023Switching Temporary Teachers for Semi-Supervised Semantic Segmentationcited by 23
  19. 2023Rethinking Data Perturbation and Model Stabilization for Semi-supervised Medical Image Segmentationcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Dual Teacher-Student Learning for Semi-supervised Medical Image Segmentation
  2. 2025Decoupled Competitive Framework for Semi-supervised Medical Image Segmentation
  3. 2025Adaptive Knowledge Transferring with Switching Dual-Student Framework for Semi-Supervised Medical Image Segmentation
  4. 2025Semi-Supervised Biomedical Image Segmentation via Diffusion Models and Teacher-Student Co-Training
  5. 2023Competitive Ensembling Teacher-Student Framework for Semi-Supervised Left Atrium MRI Segmentation
  6. 2025A Unified Framework for Semi-Supervised Image Segmentation and Registration
  7. 2026Addressing data annotation scarcity in Brain Tumor Segmentation on 3D MRI scan Using a Semi-Supervised Teacher-Student Framework
  8. 2024Uncertainty-Guided Cross Attention Ensemble Mean Teacher for Semi-supervised Medical Image Segmentation