# Dual-Teacher++: Exploiting Intra-domain and Inter-domain Knowledge with Reliable Transfer for Cardiac Segmentation

arXiv: https://arxiv.org/abs/2101.02375 · HTML: https://mlsanity.com/paper/2101.02375/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 62 references that are in the map.
- First posted on arXiv 7 Jan 2021.

## Builds on (most cited first, 30 of 42)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [Uncertainty-aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentation](https://mlsanity.com/paper/1907.07034/) (2019)
- [Semi-supervised Domain Adaptation via Minimax Entropy](https://mlsanity.com/paper/1904.06487/) (2019)
- [GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification](https://mlsanity.com/paper/1803.01229/) (2018)
- [Characterizing and Avoiding Negative Transfer](https://mlsanity.com/paper/1811.09751/) (2018)
- [Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding](https://mlsanity.com/paper/1511.02680/) (2015)
- [Large scale distributed neural network training through online distillation](https://mlsanity.com/paper/1804.03235/) (2018)
- [Transformation Consistent Self-ensembling Model for Semi-supervised Medical Image Segmentation](https://mlsanity.com/paper/1903.00348/) (2019)
- [Data augmentation using learned transformations for one-shot medical image segmentation](https://mlsanity.com/paper/1902.09383/) (2019)
- [Harmonizing Transferability and Discriminability for Adapting Object Detectors](https://mlsanity.com/paper/2003.06297/) (2020)
- [Synergistic Image and Feature Adaptation: Towards Cross-Modality Domain Adaptation for Medical Image Segmentation](https://mlsanity.com/paper/1901.08211/) (2019)
- [Evaluation of Algorithms for Multi-Modality Whole Heart Segmentation: An Open-Access Grand Challenge](https://mlsanity.com/paper/1902.07880/) (2019)
- [Unsupervised domain adaptation for medical imaging segmentation with self-ensembling](https://mlsanity.com/paper/1811.06042/) (2018)
- [Importance Weighted Adversarial Nets for Partial Domain Adaptation](https://mlsanity.com/paper/1803.09210/) (2018)
- [Unpaired Multi-modal Segmentation via Knowledge Distillation](https://mlsanity.com/paper/2001.03111/) (2020)
- [Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation](https://mlsanity.com/paper/2006.16806/) (2020)
- [Active Adversarial Domain Adaptation](https://mlsanity.com/paper/1904.07848/) (2019)
- [SynSeg-Net: Synthetic Segmentation Without Target Modality Ground Truth](https://mlsanity.com/paper/1810.06498/) (2018)
- [Transfer Learning for Domain Adaptation in MRI: Application in Brain Lesion Segmentation](https://mlsanity.com/paper/1702.07841/) (2017)
- [Incorporating Temporal Prior from Motion Flow for Instrument Segmentation in Minimally Invasive Surgery Video](https://mlsanity.com/paper/1907.07899/) (2019)
- [Online Meta-Learning for Multi-Source and Semi-Supervised Domain Adaptation](https://mlsanity.com/paper/2004.04398/) (2020)
- [Unsupervised Cross-Modality Domain Adaptation of ConvNets for Biomedical Image Segmentations with Adversarial Loss](https://mlsanity.com/paper/1804.10916/) (2018)
- [Universal Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/1811.10323/) (2018)

## Built on by (most cited first, 8 of 8)

- [SimCVD: Simple Contrastive Voxel-Wise Representation Distillation for Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2108.06227/) (2021)
- [UniMiSS: Universal Medical Self-Supervised Learning via Breaking Dimensionality Barrier](https://mlsanity.com/paper/2112.09356/) (2021)
- [LE-UDA: Label-efficient unsupervised domain adaptation for medical image segmentation](https://mlsanity.com/paper/2212.02078/) (2022)
- [Deep learning for unsupervised domain adaptation in medical imaging: Recent advancements and future perspectives](https://mlsanity.com/paper/2308.01265/) (2023)
- [Multi-Modality Cardiac Image Computing: A Survey](https://mlsanity.com/paper/2208.12881/) (2022)
- [Improving Differentiable Architecture Search via Self-Distillation](https://mlsanity.com/paper/2302.05629/) (2023)
- [Navigating Distribution Shifts in Medical Image Analysis: A Survey](https://mlsanity.com/paper/2411.05824/) (2024)
- [Attentive Continuous Generative Self-training for Unsupervised Domain Adaptive Medical Image Translation](https://mlsanity.com/paper/2305.14589/) (2023)

## Abstract (arXiv)

Annotation scarcity is a long-standing problem in medical image analysis area. To efficiently leverage limited annotations, abundant unlabeled data are additionally exploited in semi-supervised learning, while well-established cross-modality data are investigated in domain adaptation. In this paper, we aim to explore the feasibility of concurrently leveraging both unlabeled data and cross-modality data for annotation-efficient cardiac segmentation. To this end, we propose a cutting-edge semi-supervised domain adaptation framework, namely Dual-Teacher++. Besides directly learning from limited labeled target domain data (e.g., CT) via a student model adopted by previous literature, we design novel dual teacher models, including an inter-domain teacher model to explore cross-modality priors from source domain (e.g., MR) and an intra-domain teacher model to investigate the knowledge beneath unlabeled target domain. In this way, the dual teacher models would transfer acquired inter- and intra-domain knowledge to the student model for further integration and exploitation. Moreover, to encourage reliable dual-domain knowledge transfer, we enhance the inter-domain knowledge transfer on the samples with higher similarity to target domain after appearance alignment, and also strengthen intra-domain knowledge transfer of unlabeled target data with higher prediction confidence. In this way, the student model can obtain reliable dual-domain knowledge and yield improved performance on target domain data. We extensively evaluated the feasibility of our method on the MM-WHS 2017 challenge dataset. The experiments have demonstrated the superiority of our framework over other semi-supervised learning and domain adaptation methods. Moreover, our performance gains could be yielded in bidirections,i.e., adapting from MR to CT, and from CT to MR.