# Learning to Train with Synthetic Humans

arXiv: https://arxiv.org/abs/1908.00967 · HTML: https://mlsanity.com/paper/1908.00967/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Aug 2019.

## Built on by (most cited first, 14 of 14)

- [BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion](https://mlsanity.com/paper/2306.16940/) (2023)
- [AGORA: Avatars in Geography Optimized for Regression Analysis](https://mlsanity.com/paper/2104.14643/) (2021)
- [LEAP: Learning Articulated Occupancy of People](https://mlsanity.com/paper/2104.06849/) (2021)
- [Playing for 3D Human Recovery](https://mlsanity.com/paper/2110.07588/) (2021)
- [UnrealEgo: A New Dataset for Robust Egocentric 3D Human Motion Capture](https://mlsanity.com/paper/2208.01633/) (2022)
- [Accurate 3D Body Shape Regression using Metric and Semantic Attributes](https://mlsanity.com/paper/2206.07036/) (2022)
- [Synthetic Data in Human Analysis: A Survey](https://mlsanity.com/paper/2208.09191/) (2022)
- [3D Segmentation of Humans in Point Clouds with Synthetic Data](https://mlsanity.com/paper/2212.00786/) (2022)
- [Global Adaptation meets Local Generalization: Unsupervised Domain Adaptation for 3D Human Pose Estimation](https://mlsanity.com/paper/2303.16456/) (2023)
- [4DHumanOutfit: a multi-subject 4D dataset of human motion sequences in varying outfits exhibiting large displacements](https://mlsanity.com/paper/2306.07399/) (2023)
- [A Benchmark of Video-Based Clothes-Changing Person Re-Identification](https://mlsanity.com/paper/2211.11165/) (2022)
- [ShapeBoost: Boosting Human Shape Estimation with Part-Based Parameterization and Clothing-Preserving Augmentation](https://mlsanity.com/paper/2403.01345/) (2024)
- [Human Pose Estimation in Monocular Omnidirectional Top-View Images](https://mlsanity.com/paper/2304.08186/) (2023)
- [Noise-in, Bias-out: Balanced and Real-time MoCap Solving](https://mlsanity.com/paper/2309.14330/) (2023)

## Abstract (arXiv)

Neural networks need big annotated datasets for training. However, manual annotation can be too expensive or even unfeasible for certain tasks, like multi-person 2D pose estimation with severe occlusions. A remedy for this is synthetic data with perfect ground truth. Here we explore two variations of synthetic data for this challenging problem; a dataset with purely synthetic humans and a real dataset augmented with synthetic humans. We then study which approach better generalizes to real data, as well as the influence of virtual humans in the training loss. Using the augmented dataset, without considering synthetic humans in the loss, leads to the best results. We observe that not all synthetic samples are equally informative for training, while the informative samples are different for each training stage. To exploit this observation, we employ an adversarial student-teacher framework; the teacher improves the student by providing the hardest samples for its current state as a challenge. Experiments show that the student-teacher framework outperforms normal training on the purely synthetic dataset.