# Knowing the Distance: Understanding the Gap Between Synthetic and Real Data For Face Parsing

arXiv: https://arxiv.org/abs/2303.15219 · HTML: https://mlsanity.com/paper/2303.15219/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 28 references that are in the map.
- First posted on arXiv 27 Mar 2023.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1703.10593/) (2017)
- [Deep Visual Domain Adaptation: A Survey](https://mlsanity.com/paper/1802.03601/) (2018)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [Albumentations: fast and flexible image augmentations](https://mlsanity.com/paper/1809.06839/) (2018)
- [DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort](https://mlsanity.com/paper/2104.06490/) (2021)
- [Fake It Till You Make It: Face analysis in the wild using synthetic data alone](https://mlsanity.com/paper/2109.15102/) (2021)
- [General Facial Representation Learning in a Visual-Linguistic Manner](https://mlsanity.com/paper/2112.03109/) (2021)
- [Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchy](https://mlsanity.com/paper/1912.07726/) (2019)
- [Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure](https://mlsanity.com/paper/2010.13561/) (2020)
- [Meta-Sim: Learning to Generate Synthetic Datasets](https://mlsanity.com/paper/1904.11621/) (2019)
- [DigiFace-1M: 1 Million Digital Face Images for Face Recognition](https://mlsanity.com/paper/2210.02579/) (2022)
- [Semantic-aware Grad-GAN for Virtual-to-Real Urban Scene Adaption](https://mlsanity.com/paper/1801.01726/) (2018)
- [RoI Tanh-polar Transformer Network for Face Parsing in the Wild](https://mlsanity.com/paper/2102.02717/) (2021)
- [AGRNet: Adaptive Graph Representation Learning and Reasoning for Face Parsing](https://mlsanity.com/paper/2101.07034/) (2021)
- [Hands-Up: Leveraging Synthetic Data for Hands-On-Wheel Detection](https://mlsanity.com/paper/2206.00148/) (2022)

## Built on by (most cited first, 4 of 4)

- [Learning Human Action Recognition Representations Without Real Humans](https://mlsanity.com/paper/2311.06231/) (2023)
- [Face Recognition Using Synthetic Face Data](https://mlsanity.com/paper/2305.10079/) (2023)
- [Synthetic Data Augmentation for Enhanced Chicken Carcass Instance Segmentation](https://mlsanity.com/paper/2507.18558/) (2025)
- [A Real-Calibrated Synthetic-First Data Engine](https://mlsanity.com/paper/2605.09699/) (2026)

## Abstract (arXiv)

The use of synthetic data for training computer vision algorithms has become increasingly popular due to its cost-effectiveness, scalability, and ability to provide accurate multi-modality labels. Although recent studies have demonstrated impressive results when training networks solely on synthetic data, there remains a performance gap between synthetic and real data that is commonly attributed to lack of photorealism. The aim of this study is to investigate the gap in greater detail for the face parsing task. We differentiate between three types of gaps: distribution gap, label gap, and photorealism gap. Our findings show that the distribution gap is the largest contributor to the performance gap, accounting for over 50% of the gap. By addressing this gap and accounting for the labels gap, we demonstrate that a model trained on synthetic data achieves comparable results to one trained on a similar amount of real data. This suggests that synthetic data is a viable alternative to real data, especially when real data is limited or difficult to obtain. Our study highlights the importance of content diversity in synthetic datasets and challenges the notion that the photorealism gap is the most critical factor affecting the performance of computer vision models trained on synthetic data.