# Learning from Synthetic Humans

arXiv: https://arxiv.org/abs/1701.01370 · HTML: https://mlsanity.com/paper/1701.01370/

## Key facts

- Cited by 359 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jan 2017.

## Built on by (most cited first, 30 of 359)

- [Synthetic Data from Diffusion Models Improves ImageNet Classification](https://mlsanity.com/paper/2304.08466/) (2023)
- [HuMoR: 3D Human Motion Model for Robust Pose Estimation](https://mlsanity.com/paper/2105.04668/) (2021)
- [BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion](https://mlsanity.com/paper/2306.16940/) (2023)
- [A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose](https://mlsanity.com/paper/2102.06199/) (2021)
- [AGORA: Avatars in Geography Optimized for Regression Analysis](https://mlsanity.com/paper/2104.14643/) (2021)
- [Fake It Till You Make It: Face analysis in the wild using synthetic data alone](https://mlsanity.com/paper/2109.15102/) (2021)
- [Neural Articulated Radiance Field](https://mlsanity.com/paper/2104.03110/) (2021)
- [Embodied Hands: Modeling and Capturing Hands and Bodies Together](https://mlsanity.com/paper/2201.02610/) (2022)
- [StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learners](https://mlsanity.com/paper/2306.00984/) (2023)
- [Fake it till you make it: Learning transferable representations from synthetic ImageNet clones](https://mlsanity.com/paper/2212.08420/) (2022)
- [InstantAvatar: Learning Avatars from Monocular Video in 60 Seconds](https://mlsanity.com/paper/2212.10550/) (2022)
- [SPEC: Seeing People in the Wild with an Estimated Camera](https://mlsanity.com/paper/2110.00620/) (2021)
- [Generative Models as a Data Source for Multiview Representation Learning](https://mlsanity.com/paper/2106.05258/) (2021)
- [Recovering 3D Human Mesh from Monocular Images: A Survey](https://mlsanity.com/paper/2203.01923/) (2022)
- [Generative Neural Articulated Radiance Fields](https://mlsanity.com/paper/2206.14314/) (2022)
- [Multi-HMR: Multi-Person Whole-Body Human Mesh Recovery in a Single Shot](https://mlsanity.com/paper/2402.14654/) (2024)
- [Pre-training without Natural Images](https://mlsanity.com/paper/2101.08515/) (2021)
- [EMDB: The Electromagnetic Database of Global 3D Human Pose and Shape in the Wild](https://mlsanity.com/paper/2308.16894/) (2023)
- [PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation](https://mlsanity.com/paper/2105.02465/) (2021)
- [NeuroMorph: Unsupervised Shape Interpolation and Correspondence in One Go](https://mlsanity.com/paper/2106.09431/) (2021)
- [SCALE: Modeling Clothed Humans with a Surface Codec of Articulated Local Elements](https://mlsanity.com/paper/2104.07660/) (2021)
- [Learning Vision from Models Rivals Learning Vision from Data](https://mlsanity.com/paper/2312.17742/) (2023)
- [High-fidelity 3D Human Digitization from Single 2K Resolution Images](https://mlsanity.com/paper/2303.15108/) (2023)
- [MOTSynth: How Can Synthetic Data Help Pedestrian Detection and Tracking?](https://mlsanity.com/paper/2108.09518/) (2021)
- [Human-Art: A Versatile Human-Centric Dataset Bridging Natural and Artificial Scenes](https://mlsanity.com/paper/2303.02760/) (2023)
- [Unsupervised Learning of Efficient Geometry-Aware Neural Articulated Representations](https://mlsanity.com/paper/2204.08839/) (2022)
- [Playing for 3D Human Recovery](https://mlsanity.com/paper/2110.07588/) (2021)
- [SINC: Spatial Composition of 3D Human Motions for Simultaneous Action Generation](https://mlsanity.com/paper/2304.10417/) (2023)
- [SynthCLIP: Are We Ready for a Fully Synthetic CLIP Training?](https://mlsanity.com/paper/2402.01832/) (2024)
- [HumanVid: Demystifying Training Data for Camera-controllable Human Image Animation](https://mlsanity.com/paper/2407.17438/) (2024)

## Abstract (arXiv)

Estimating human pose, shape, and motion from images and videos are fundamental challenges with many applications. Recent advances in 2D human pose estimation use large amounts of manually-labeled training data for learning convolutional neural networks (CNNs). Such data is time consuming to acquire and difficult to extend. Moreover, manual labeling of 3D pose, depth and motion is impractical. In this work we present SURREAL (Synthetic hUmans foR REAL tasks): a new large-scale dataset with synthetically-generated but realistic images of people rendered from 3D sequences of human motion capture data. We generate more than 6 million frames together with ground truth pose, depth maps, and segmentation masks. We show that CNNs trained on our synthetic dataset allow for accurate human depth estimation and human part segmentation in real RGB images. Our results and the new dataset open up new possibilities for advancing person analysis using cheap and large-scale synthetic data.