# Transferring and Regularizing Prediction for Semantic Segmentation

arXiv: https://arxiv.org/abs/2006.06570 · HTML: https://mlsanity.com/paper/2006.06570/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jun 2020.

## Built on by (most cited first, 17 of 17)

- [Self-supervised Augmentation Consistency for Adapting Semantic Segmentation](https://mlsanity.com/paper/2105.00097/) (2021)
- [Generalize then Adapt: Source-Free Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2108.11249/) (2021)
- [Re-distributing Biased Pseudo Labels for Semi-supervised Semantic Segmentation: A Baseline Investigation](https://mlsanity.com/paper/2107.11279/) (2021)
- [Survey on Unsupervised Domain Adaptation for Semantic Segmentation for Visual Perception in Automated Driving](https://mlsanity.com/paper/2304.11928/) (2023)
- [Semantic Image Segmentation: Two Decades of Research](https://mlsanity.com/paper/2302.06378/) (2023)
- [On the Road to Online Adaptation for Semantic Image Segmentation](https://mlsanity.com/paper/2203.16195/) (2022)
- [Unsupervised Domain Adaptation for Semantic Image Segmentation: a Comprehensive Survey](https://mlsanity.com/paper/2112.03241/) (2021)
- [A Curriculum-style Self-training Approach for Source-Free Semantic Segmentation](https://mlsanity.com/paper/2106.11653/) (2021)
- [Unsupervised Contrastive Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2204.08399/) (2022)
- [TransAdapt: A Transformative Framework for Online Test Time Adaptive Semantic Segmentation](https://mlsanity.com/paper/2302.14611/) (2023)
- [Transformer-Unet: Raw Image Processing with Unet](https://mlsanity.com/paper/2109.08417/) (2021)
- [SUPRA: Superpixel Guided Loss for Improved Multi-modal Segmentation in Endoscopy](https://mlsanity.com/paper/2211.04658/) (2022)
- [Explaining Cross-Domain Recognition with Interpretable Deep Classifier](https://mlsanity.com/paper/2211.08249/) (2022)
- [Domain Generalization for Endoscopic Image Segmentation by Disentangling Style-Content Information and SuperPixel Consistency](https://mlsanity.com/paper/2409.12450/) (2024)
- [HazardNet: Road Debris Detection by Augmentation of Synthetic Models](https://mlsanity.com/paper/2303.07547/) (2023)
- [Regularizing Self-training for Unsupervised Domain Adaptation via Structural Constraints](https://mlsanity.com/paper/2305.00131/) (2023)
- [SP 3 : Superpixel-propagated pseudo-label learning for weakly semi-supervised medical image segmentation](https://mlsanity.com/paper/2411.11636/) (2024)

## Abstract (arXiv)

Semantic segmentation often requires a large set of images with pixel-level annotations. In the view of extremely expensive expert labeling, recent research has shown that the models trained on photo-realistic synthetic data (e.g., computer games) with computer-generated annotations can be adapted to real images. Despite this progress, without constraining the prediction on real images, the models will easily overfit on synthetic data due to severe domain mismatch. In this paper, we novelly exploit the intrinsic properties of semantic segmentation to alleviate such problem for model transfer. Specifically, we present a Regularizer of Prediction Transfer (RPT) that imposes the intrinsic properties as constraints to regularize model transfer in an unsupervised fashion. These constraints include patch-level, cluster-level and context-level semantic prediction consistencies at different levels of image formation. As the transfer is label-free and data-driven, the robustness of prediction is addressed by selectively involving a subset of image regions for model regularization. Extensive experiments are conducted to verify the proposal of RPT on the transfer of models trained on GTA5 and SYNTHIA (synthetic data) to Cityscapes dataset (urban street scenes). RPT shows consistent improvements when injecting the constraints on several neural networks for semantic segmentation. More remarkably, when integrating RPT into the adversarial-based segmentation framework, we report to-date the best results: mIoU of 53.2%/51.7% when transferring from GTA5/SYNTHIA to Cityscapes, respectively.