# Contrastive Syn-to-Real Generalization

arXiv: https://arxiv.org/abs/2104.02290 · HTML: https://mlsanity.com/paper/2104.02290/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 42 references that are in the map.
- First posted on arXiv 6 Apr 2021.

## Builds on (most cited first, 30 of 38)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [Self-Supervised Learning of Pretext-Invariant Representations](https://mlsanity.com/paper/1912.01991/) (2019)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Contrastive Representation Distillation](https://mlsanity.com/paper/1910.10699/) (2019)
- [Data-Efficient Image Recognition with Contrastive Predictive Coding](https://mlsanity.com/paper/1905.09272/) (2019)
- [Domain Generalization via Invariant Feature Representation](https://mlsanity.com/paper/1301.2115/) (2013)
- [VisDA: The Visual Domain Adaptation Challenge](https://mlsanity.com/paper/1710.06924/) (2017)
- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net](https://mlsanity.com/paper/1807.09441/) (2018)

## Built on by (most cited first, 30 of 32)

- [SAT: Dynamic Spatial Aptitude Training for Multimodal Language Models](https://mlsanity.com/paper/2412.07755/) (2024)
- [Label-Free Liver Tumor Segmentation](https://mlsanity.com/paper/2303.14869/) (2023)
- [Siamese Contrastive Embedding Network for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2206.14475/) (2022)
- [Pin the Memory: Learning to Generalize Semantic Segmentation](https://mlsanity.com/paper/2204.03609/) (2022)
- [Source-Free Open Compound Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/2106.03422/) (2021)
- [PASTA: Proportional Amplitude Spectrum Training Augmentation for Syn-to-Real Domain Generalization](https://mlsanity.com/paper/2212.00979/) (2022)
- [Behavioral Intention Prediction in Driving Scenes: A Survey](https://mlsanity.com/paper/2211.00385/) (2022)
- [Texture Learning Domain Randomization for Domain Generalized Segmentation](https://mlsanity.com/paper/2303.11546/) (2023)
- [Few-Shot Adaptation of Pre-Trained Networks for Domain Shift](https://mlsanity.com/paper/2205.15234/) (2022)
- [A Scaling Law for Synthetic-to-Real Transfer: How Much Is Your Pre-training Effective?](https://mlsanity.com/paper/2108.11018/) (2021)
- [Source-Free Domain Adaptation Guided by Vision and Vision-Language Pre-Training](https://mlsanity.com/paper/2405.02954/) (2024)
- [Rethinking the Role of Pre-Trained Networks in Source-Free Domain Adaptation](https://mlsanity.com/paper/2212.07585/) (2022)
- [Generalized Semantic Segmentation by Self-Supervised Source Domain Projection and Multi-Level Contrastive Learning](https://mlsanity.com/paper/2303.01906/) (2023)
- [A New Benchmark: On the Utility of Synthetic Data with Blender for Bare Supervised Learning and Downstream Domain Adaptation](https://mlsanity.com/paper/2303.09165/) (2023)
- [Pick the Best Pre-trained Model: Towards Transferability Estimation for Medical Image Segmentation](https://mlsanity.com/paper/2307.11958/) (2023)
- [Self-Supervised Learning of Domain Invariant Features for Depth Estimation](https://mlsanity.com/paper/2106.02594/) (2021)
- [Adversarial Semantic Hallucination for Domain Generalized Semantic Segmentation](https://mlsanity.com/paper/2106.04144/) (2021)
- [Semantic Self-adaptation: Enhancing Generalization with a Single Sample](https://mlsanity.com/paper/2208.05788/) (2022)
- [SUG: Single-dataset Unified Generalization for 3D Point Cloud Classification](https://mlsanity.com/paper/2305.09160/) (2023)
- [BELT:Bootstrapping Electroencephalography-to-Language Decoding and Zero-Shot Sentiment Classification by Natural Language Supervision](https://mlsanity.com/paper/2309.12056/) (2023)
- [Reducing the Amount of Real World Data for Object Detector Training with Synthetic Data](https://mlsanity.com/paper/2202.00632/) (2022)
- [TRoVE: Transforming Road Scene Datasets into Photorealistic Virtual Environments](https://mlsanity.com/paper/2208.07943/) (2022)
- [Feature diversity in self-supervised learning](https://mlsanity.com/paper/2209.01275/) (2022)
- [Mind the Backbone: Minimizing Backbone Distortion for Robust Object Detection](https://mlsanity.com/paper/2303.14744/) (2023)
- [Learning Data-Efficient and Generalizable Neural Operators via Fundamental Physics Knowledge](https://mlsanity.com/paper/2602.15184/) (2026)
- [GCISG: Guided Causal Invariant Learning for Improved Syn-to-real Generalization](https://mlsanity.com/paper/2208.10024/) (2022)
- [Adversarial training of Keyword Spotting to Minimize TTS Data Overfitting](https://mlsanity.com/paper/2408.10463/) (2024)
- [P-WAE: Generalized Patch-Wasserstein Autoencoder for Anomaly Screening](https://mlsanity.com/paper/2108.03815/) (2021)
- [Few-Max: Few-Shot Domain Adaptation for Unsupervised Contrastive Representation Learning](https://mlsanity.com/paper/2206.10137/) (2022)
- [Generalization Beyond Feature Alignment: Concept Activation-Guided Contrastive Learning](https://mlsanity.com/paper/2211.06843/) (2022)

## Abstract (arXiv)

Training on synthetic data can be beneficial for label or data-scarce scenarios. However, synthetically trained models often suffer from poor generalization in real domains due to domain gaps. In this work, we make a key observation that the diversity of the learned feature embeddings plays an important role in the generalization performance. To this end, we propose contrastive synthetic-to-real generalization (CSG), a novel framework that leverages the pre-trained ImageNet knowledge to prevent overfitting to the synthetic domain, while promoting the diversity of feature embeddings as an inductive bias to improve generalization. In addition, we enhance the proposed CSG framework with attentional pooling (A-pool) to let the model focus on semantically important regions and further improve its generalization. We demonstrate the effectiveness of CSG on various synthetic training tasks, exhibiting state-of-the-art performance on zero-shot domain generalization.