mlsanityData to 14 Oct 2026

DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort

arXiv 2104.06490 ↗First posted 13 Apr 2021cs.CVcited by 264

Not on the map. Why?

Cited by 67 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce DatasetGAN: an automatic procedure to generate massive datasets of high-quality semantically segmented images requiring minimal human effort. Current deep networks are extremely data-hungry, benefiting from training on large-scale datasets, which are time consuming to annotate. Our method relies on the power of recent GANs to generate realistic images. We show how the GAN latent code can be decoded to produce a semantic segmentation of the image. Training the decoder only needs a few labeled examples to generalize to the rest of the latent space, resulting in an infinite annotated dataset generator! These generated datasets can then be used for training any computer vision architecture just as real datasets are. As only a few images need to be manually segmented, it becomes possible to annotate images in extreme detail and generate datasets with rich object and part segmentations. To showcase the power of our approach, we generated datasets for 7 image segmentation tasks which include pixel-level labels for 34 human face parts, and 32 car parts. Our approach outperforms all semi-supervised baselines significantly and is on par with fully supervised methods, which in some cases require as much as 100x more annotated data as our method.

Key facts. Cited by 264 papers in the map (data through 14 Oct 2026). Builds on 40 of its 59 references that are in the map. First posted on arXiv 13 Apr 2021.

Builds on

40 of 59 references.

References · 40

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  8. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  9. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  10. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  11. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  12. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
28 more
  1. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  2. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
  3. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,391
  4. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  5. 2019Contrastive Multiview Codingcited by 1,301
  6. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,069
  7. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 808
  8. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 734
  9. 2018ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentationcited by 623
  10. 2017Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning resultscited by 472
  11. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 455
  12. 2019Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderercited by 221
  13. 2018Adversarial Learning for Semi-Supervised Semantic Segmentationcited by 173
  14. 2017Image to Image Translation for Domain Adaptationcited by 171
  15. 2019Semi-Supervised Semantic Segmentation with High- and Low-level Consistencycited by 147
  16. 2021Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalizationcited by 141
  17. 2019Meta-Sim: Learning to Generate Synthetic Datasetscited by 128
  18. 2019Self-Ensembling with GAN-based Data Augmentation for Domain Adaptation in Semantic Segmentationcited by 126
  19. 2020Guided Collaborative Training for Pixel-wise Semi-Supervised Learningcited by 120
  20. 2020Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Renderingcited by 101
  21. 2018Multi-modal Cycle-consistent Generalized Zero-Shot Learningcited by 87
  22. 2016Semantic Segmentation using Adversarial Networkscited by 73
  23. 2020Meta-Sim2: Unsupervised Learning of Scene Structure for Synthetic Data Generationcited by 58
  24. 2017Generating Visual Representations for Zero-Shot Classificationcited by 37
  25. 2017Active Learning for Convolutional Neural Networks: A Core-Set Approachcited by 36
  26. 2018Cost-Sensitive Active Learning for Intracranial Hemorrhage Detectioncited by 28
  27. 2020Fed-Sim: Federated Simulation for Medical Imagingcited by 14
  28. 2020Learning High-Resolution Domain-Specific Representations with a GAN Generatorcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 18; 2022: 56; 2023: 91; 2024: 50; 2025: 33; 2026: 16.

264 papers.

Sort:

By area:

  1. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 499
  2. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  3. 2023Emergent Correspondence from Image Diffusioncited by 446
  4. 2022Kubric: A scalable dataset generatorcited by 381
  5. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 375
  6. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 327
  7. 2022Is synthetic data from generative models ready for image recognition?cited by 320
  8. 2023Your Diffusion Model is Secretly a Zero-Shot Classifiercited by 306
  9. 2023Effective Data Augmentation With Diffusion Modelscited by 283
  10. 2021Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformerscited by 248
  11. 2023Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifoldcited by 219
  12. 2023DiffuMask: Synthesizing Images with Pixel-level Annotations for Semantic Segmentation Using Diffusion Modelscited by 162
38 more
  1. 2021EditGAN: High-Precision Semantic Image Editingcited by 156
  2. 2023Localizing Object-level Shape Variations with Text-to-Image Diffusion Modelscited by 142
  3. 2023DatasetDM: Synthesizing Data with Perception Annotations Using Diffusion Modelscited by 123
  4. 2022SemMAE: Semantic-Guided Masking for Learning Masked Autoencoderscited by 114
  5. 2023Dataset Diffusion: Diffusion-based Synthetic Dataset Generation for Pixel-Level Semantic Segmentationcited by 112
  6. 2021Generative Models as a Data Source for Multiview Representation Learningcited by 104
  7. 2023Diversify Your Vision Datasets with Automatic Diffusion-Based Augmentationcited by 100
  8. 2023Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitationscited by 100
  9. 2022BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotationscited by 99
  10. 2023Diffusion Model as Representation Learnercited by 95
  11. 2021Learning to See by Looking at Noisecited by 92
  12. 2021CLIP2StyleGAN: Unsupervised Extraction of StyleGAN Edit Directionscited by 78
  13. 2022Synthesizing Informative Training Samples with GANcited by 77
  14. 2023Adaptive Shells for Efficient Neural Radiance Field Renderingcited by 69
  15. 2021Mind the Gap: Domain Gap Control for Single Shot Domain Adaptation for Generative Adversarial Networkscited by 68
  16. 2023Real-Time Radiance Fields for Single-Image Portrait View Synthesiscited by 64
  17. 2021GAN-Supervised Dense Visual Alignmentcited by 62
  18. 2022Expanding Small-Scale Datasets with Guided Imaginationcited by 62
  19. 2023FreeMask: Synthetic Images with Dense Annotations Make Stronger Segmentation Modelscited by 61
  20. 2021Unsupervised Part Discovery from Contrastive Reconstructioncited by 57
  21. 2023DiffusionSeg: Adapting Diffusion Towards Unsupervised Object Discoverycited by 54
  22. 2021DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderercited by 52
  23. 2023Generative Models: What Do They Know? Do They Know Things? Let's Find Out!cited by 50
  24. 2023DGInStyle: Domain-Generalizable Semantic Segmentation with Image Diffusion Models and Stylized Semantic Controlcited by 45
  25. 2023StyleGAN knows Normal, Depth, Albedo, and Morecited by 44
  26. 2024EmerDiff: Emerging Pixel-level Semantic Knowledge in Diffusion Modelscited by 43
  27. 2024SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentationcited by 41
  28. 2023Diffusion Models and Semi-Supervised Learners Benefit Mutually with Few Labelscited by 37
  29. 2023DreamTeacher: Pretraining Image Backbones with Deep Generative Modelscited by 37
  30. 2023Real-Fake: Effective Training Data Synthesis Through Distribution Matchingcited by 36
  31. 2023MosaicFusion: Diffusion Models as Data Augmenters for Large Vocabulary Instance Segmentationcited by 35
  32. 2022State-of-the-Art in the Architecture, Methods and Applications of StyleGANcited by 34
  33. 2023Fill-Up: Balancing Long-Tailed Data with Generative Modelscited by 34
  34. 2022StyleT2I: Toward Compositional and High-Fidelity Text-to-Image Synthesiscited by 33
  35. 2023Do text-free diffusion models learn discriminative visual representations?cited by 33
  36. 2022DL4SciVis: A State-of-the-Art Survey on Deep Learning for Scientific Visualizationcited by 32
  37. 2024Beyond the Contact: Discovering Comprehensive Affordance for 3D Objects from Pre-trained 2D Diffusion Modelscited by 32
  38. 2022One-Shot Adaptation of GAN in Just One CLIPcited by 31

50 most cited of 264 All 264 (JSON)