# Margin Preserving Self-paced Contrastive Learning Towards Domain Adaptation for Medical Image Segmentation

arXiv: https://arxiv.org/abs/2103.08454 · HTML: https://mlsanity.com/paper/2103.08454/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 45 references that are in the map.
- First posted on arXiv 15 Mar 2021.

## Builds on (most cited first, 30 of 30)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Self-Supervised Learning of Pretext-Invariant Representations](https://mlsanity.com/paper/1912.01991/) (2019)
- [Maximum Classifier Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1712.02560/) (2017)
- [ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/1811.12833/) (2018)
- [Learning to Adapt Structured Output Space for Semantic Segmentation](https://mlsanity.com/paper/1802.10349/) (2018)
- [Bidirectional Learning for Domain Adaptation of Semantic Segmentation](https://mlsanity.com/paper/1904.10620/) (2019)
- [Taking A Closer Look at Domain Shift: Category-level Adversaries for Semantics Consistent Domain Adaptation](https://mlsanity.com/paper/1809.09478/) (2018)
- [Unsupervised Intra-domain Adaptation for Semantic Segmentation through Self-Supervision](https://mlsanity.com/paper/2004.07703/) (2020)
- [Revisiting Self-Training for Neural Sequence Generation](https://mlsanity.com/paper/1909.13788/) (2019)
- [Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/1910.13049/) (2019)
- [Unsupervised Bidirectional Cross-Modality Adaptation via Deeply Synergistic Image and Feature Alignment for Medical Image Segmentation](https://mlsanity.com/paper/2002.02255/) (2020)
- [Self-Training for End-to-End Speech Recognition](https://mlsanity.com/paper/1909.09116/) (2019)
- [Synergistic Image and Feature Adaptation: Towards Cross-Modality Domain Adaptation for Medical Image Segmentation](https://mlsanity.com/paper/1901.08211/) (2019)
- [No More Discrimination: Cross City Adaptation of Road Scene Segmenters](https://mlsanity.com/paper/1704.08509/) (2017)
- [Unsupervised Cross-Modality Domain Adaptation of ConvNets for Biomedical Image Segmentations with Adversarial Loss](https://mlsanity.com/paper/1804.10916/) (2018)
- [Collaborative Unsupervised Domain Adaptation for Medical Image Diagnosis](https://mlsanity.com/paper/2007.07222/) (2020)
- [Unsupervised Learning for Cell-level Visual Representation in Histopathology Images with Generative Adversarial Networks](https://mlsanity.com/paper/1711.11317/) (2017)
- [Fully Convolutional Network Ensembles for White Matter Hyperintensities Segmentation in MR Images](https://mlsanity.com/paper/1802.05203/) (2018)
- [Distribution-induced Bidirectional Generative Adversarial Network for Graph Representation Learning](https://mlsanity.com/paper/1912.01899/) (2019)

## Built on by (most cited first, 20 of 20)

- [HYperbolic Self-Paced Learning for Self-Supervised Skeleton-based Action Representations](https://mlsanity.com/paper/2303.06242/) (2023)
- [Self-Paced Contrastive Learning for Semi-supervised Medical Image Segmentation with Meta-labels](https://mlsanity.com/paper/2107.13741/) (2021)
- [MAPSeg: Unified Unsupervised Domain Adaptation for Heterogeneous Medical Image Segmentation Based on 3D Masked Autoencoding and Pseudo-Labeling](https://mlsanity.com/paper/2303.09373/) (2023)
- [Deep learning for unsupervised domain adaptation in medical imaging: Recent advancements and future perspectives](https://mlsanity.com/paper/2308.01265/) (2023)
- [Semi-supervised Domain Adaptive Medical Image Segmentation through Consistency Regularized Disentangled Contrastive Learning](https://mlsanity.com/paper/2307.02798/) (2023)
- [Mind The Gap: Alleviating Local Imbalance for Unsupervised Cross-Modality Medical Image Segmentation](https://mlsanity.com/paper/2205.11888/) (2022)
- [MSCDA: Multi-level Semantic-guided Contrast Improves Unsupervised Domain Adaptation for Breast MRI Segmentation in Small Datasets](https://mlsanity.com/paper/2301.02554/) (2023)
- [COMICS: End-to-end Bi-grained Contrastive Learning for Multi-face Forgery Detection](https://mlsanity.com/paper/2308.01520/) (2023)
- [Navigating Distribution Shifts in Medical Image Analysis: A Survey](https://mlsanity.com/paper/2411.05824/) (2024)
- [DDFP: Data-dependent Frequency Prompt for Source Free Domain Adaptation of Medical Image Segmentation](https://mlsanity.com/paper/2505.09927/) (2025)
- [Semantic-Aware Contrastive Learning for Multi-object Medical Image Segmentation](https://mlsanity.com/paper/2106.01596/) (2021)
- [ConFUDA: Contrastive Fewshot Unsupervised Domain Adaptation for Medical Image Segmentation](https://mlsanity.com/paper/2206.03888/) (2022)
- [Advancing Brain Imaging Analysis Step-by-step via Progressive Self-paced Learning](https://mlsanity.com/paper/2407.16128/) (2024)
- [ST-DAI: Single-shot 2.5D Spatial Transcriptomics with Intra-Sample Domain Adaptive Imputation for Cost-efficient 3D Reconstruction](https://mlsanity.com/paper/2507.21516/) (2025)
- [Adaptive Contrastive Learning with Dynamic Correlation for Multi-Phase Organ Segmentation](https://mlsanity.com/paper/2210.08652/) (2022)
- [Bridging the Inter-Domain Gap through Low-Level Features for Cross-Modal Medical Image Segmentation](https://mlsanity.com/paper/2505.11909/) (2025)
- [Medical Image Classification Using Transfer Learning and Chaos Game Optimization on the Internet of Medical Things](https://mlsanity.com/paper/2312.07437/) (2023)
- [Unsupervised Domain Adaptation via Similarity-based Prototypes for Cross-Modality Segmentation](https://mlsanity.com/paper/2510.20596/) (2025)
- [TCSA-UDA: Text-Driven Cross-Semantic Alignment for Unsupervised Domain Adaptation in Medical Image Segmentation](https://mlsanity.com/paper/2511.05782/) (2025)
- [Semantic-aware Random Convolution and Source Matching for Domain Generalization in Medical Image Segmentation](https://mlsanity.com/paper/2512.01510/) (2025)

## Abstract (arXiv)

To bridge the gap between the source and target domains in unsupervised domain adaptation (UDA), the most common strategy puts focus on matching the marginal distributions in the feature space through adversarial learning. However, such category-agnostic global alignment lacks of exploiting the class-level joint distributions, causing the aligned distribution less discriminative. To address this issue, we propose in this paper a novel margin preserving self-paced contrastive Learning (MPSCL) model for cross-modal medical image segmentation. Unlike the conventional construction of contrastive pairs in contrastive learning, the domain-adaptive category prototypes are utilized to constitute the positive and negative sample pairs. With the guidance of progressively refined semantic prototypes, a novel margin preserving contrastive loss is proposed to boost the discriminability of embedded representation space. To enhance the supervision for contrastive learning, more informative pseudo-labels are generated in target domain in a self-paced way, thus benefiting the category-aware distribution alignment for UDA. Furthermore, the domain-invariant representations are learned through joint contrastive learning between the two domains. Extensive experiments on cross-modal cardiac segmentation tasks demonstrate that MPSCL significantly improves semantic segmentation performance, and outperforms a wide variety of state-of-the-art methods by a large margin.