# RenderGAN: Generating Realistic Labeled Data

arXiv: https://arxiv.org/abs/1611.01331 · HTML: https://mlsanity.com/paper/1611.01331/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Nov 2016.

## Built on by (most cited first, 20 of 20)

- [AutoFlow: Learning a Better Training Set for Optical Flow](https://mlsanity.com/paper/2104.14544/) (2021)
- [Ensembling with Deep Generative Views](https://mlsanity.com/paper/2104.14551/) (2021)
- [A survey of synthetic data augmentation methods in computer vision](https://mlsanity.com/paper/2403.10075/) (2024)
- [A Comprehensive Overview and Comparative Analysis on Deep Learning Models: CNN, RNN, LSTM, GRU](https://mlsanity.com/paper/2305.17473/) (2023)
- [Mitigating Data Heterogeneity in Federated Learning with Data Augmentation](https://mlsanity.com/paper/2206.09979/) (2022)
- [LARGE: Latent-Based Regression through GAN Semantics](https://mlsanity.com/paper/2107.11186/) (2021)
- [Distilling and Transferring Knowledge via cGAN-generated Samples for Image Classification and Regression](https://mlsanity.com/paper/2104.03164/) (2021)
- [In What Ways Are Deep Neural Networks Invariant and How Should We Measure This?](https://mlsanity.com/paper/2210.03773/) (2022)
- [Can You Learn to See Without Images? Procedural Warm-Up for Vision Transformers](https://mlsanity.com/paper/2511.13945/) (2025)
- [Making Invisible Visible: Data-Driven Seismic Inversion with Spatio-temporally Constrained Data Augmentation](https://mlsanity.com/paper/2106.11892/) (2021)
- [Enforcing Morphological Information in Fully Convolutional Networks to Improve Cell Instance Segmentation in Fluorescence Microscopy Images](https://mlsanity.com/paper/2106.05843/) (2021)
- [Quantifying point cloud realism through adversarially learned latent representations](https://mlsanity.com/paper/2109.11775/) (2021)
- [Fully-Synthetic Training for Visual Quality Inspection in Automotive Production](https://mlsanity.com/paper/2503.09354/) (2025)
- [Automatic Generation of Machine Learning Synthetic Data Using ROS](https://mlsanity.com/paper/2106.04547/) (2021)
- [ParticleAugment: Sampling-Based Data Augmentation](https://mlsanity.com/paper/2106.08693/) (2021)
- [Knowledge-Guided Data-Centric AI in Healthcare: Progress, Shortcomings, and Future Directions](https://mlsanity.com/paper/2212.13591/) (2022)
- [Evaluating Machine Learning Models with NERO: Non-Equivariance Revealed on Orbits](https://mlsanity.com/paper/2305.19889/) (2023)
- [PanoMixSwap Panorama Mixing via Structural Swapping for Indoor Scene Understanding](https://mlsanity.com/paper/2309.09514/) (2023)
- [A Label Propagation Strategy for CutMix in Multi-Label Remote Sensing Image Classification](https://mlsanity.com/paper/2405.13451/) (2024)
- [InAugment: Improving Classifiers via Internal Augmentation](https://mlsanity.com/paper/2104.03843/) (2021)

## Abstract (arXiv)

Deep Convolutional Neuronal Networks (DCNNs) are showing remarkable performance on many computer vision tasks. Due to their large parameter space, they require many labeled samples when trained in a supervised setting. The costs of annotating data manually can render the use of DCNNs infeasible. We present a novel framework called RenderGAN that can generate large amounts of realistic, labeled images by combining a 3D model and the Generative Adversarial Network framework. In our approach, image augmentations (e.g. lighting, background, and detail) are learned from unlabeled data such that the generated images are strikingly realistic while preserving the labels known from the 3D model. We apply the RenderGAN framework to generate images of barcode-like markers that are attached to honeybees. Training a DCNN on data generated by the RenderGAN yields considerably better performance than training it on various baselines.