Data to 7 Oct 2026

Learning from Simulated and Unsupervised Images through Adversarial Training

arXiv 1612.07828 ↗First posted 22 Dec 2016cs.CV cs.LG cs.NEcited by 287

Not on the map. Why?

Abstract (arXiv)

With recent progress in graphics, it has become more tractable to train models on synthetic images, potentially avoiding the need for expensive annotations. However, learning from synthetic images may not achieve the desired performance due to a gap between synthetic and real image distributions. To reduce this gap, we propose Simulated+Unsupervised (S+U) learning, where the task is to learn a model to improve the realism of a simulator's output using unlabeled real data, while preserving the annotation information from the simulator. We develop a method for S+U learning that uses an adversarial network similar to Generative Adversarial Networks (GANs), but with synthetic images as inputs instead of random vectors. We make several key modifications to the standard GAN algorithm to preserve annotations, avoid artifacts, and stabilize training: (i) a 'self-regularization' term, (ii) a local adversarial loss, and (iii) updating the discriminator using a history of refined images. We show that this enables generation of highly realistic images, which we demonstrate both qualitatively and with a user study. We quantitatively evaluate the generated images by training models for gaze estimation and hand pose estimation. We show a significant improvement over using synthetic images, and achieve state-of-the-art results on the MPIIGaze dataset without any labeled real data.

Key facts. Cited by 287 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Dec 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 117; 2022: 59; 2023: 45; 2024: 31; 2025: 22; 2026: 12.

287 papers.

Sort:

By area:

2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 322
2023TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinementcited by 281
2022TAP-Vid: A Benchmark for Tracking Any Point in a Videocited by 258
2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
2021Fake It Till You Make It: Face analysis in the wild using synthetic data alonecited by 200
2021Adversarial Examples Make Strong Poisonscited by 115
2024One-Step Image Translation with Text-to-Image Modelscited by 111
2021Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Controlcited by 106
2021Generative Models as a Data Source for Multiview Representation Learningcited by 104
2021Pre-training without Natural Imagescited by 100
2022Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectivescited by 98

38 more

2022A Comprehensive Survey of Image Augmentation Techniques for Deep Learningcited by 95
2021Invariant Information Bottleneck for Domain Generalizationcited by 86
2021Weakly Supervised Object Localization and Detection: A Surveycited by 84
2021Prototypical Cross-domain Self-supervised Learning for Few-shot Unsupervised Domain Adaptationcited by 83
2021Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmarkcited by 82
2021Diverse Image Inpainting with Bidirectional and Autoregressive Transformerscited by 80
2021Enhancing Photorealism Enhancementcited by 74
2021DG-Font: Deformable Generative Networks for Unsupervised Font Generationcited by 70
2021The Spatially-Correlative Loss for Various Image Translation Taskscited by 66
2022Transferability in Deep Learning: A Surveycited by 57
2021Ensembling with Deep Generative Viewscited by 53
2021Multimodal Image Synthesis and Editing: The Generative AI Eracited by 52
2022Semi-Supervised and Unsupervised Deep Visual Learning: A Surveycited by 52
2021On Aliased Resizing and Surprising Subtleties in GAN Evaluationcited by 51
2023Training on Thin Air: Improve Image Classification with Generated Datacited by 50
2022AutoBalance: Optimized Loss Functions for Imbalanced Datacited by 49
2021Finding an Unsupervised Image Segmenter in Each of Your Deep Generative Modelscited by 44
2021Tune it the Right Way: Unsupervised Validation of Domain Adaptation via Soft Neighborhood Densitycited by 44
2021Generalizing Gaze Estimation with Outlier-guided Collaborative Adaptationcited by 42
2021Unbalanced Feature Transport for Exemplar-based Image Translationcited by 41
2021Deep Symmetric Adaptation Network for Cross-modality Medical Image Segmentationcited by 39
2021Sketch Your Own GANcited by 37
2021Bridging the Gap between Events and Frames through Unsupervised Domain Adaptationcited by 36
2024A survey of synthetic data augmentation methods in computer visioncited by 34
2025Model Adaptation: Unsupervised Domain Adaptation without Source Datacited by 34
2021Automatic Gaze Analysis: A Survey of Deep Learning based Approachescited by 33
2021LabOR: Labeling Only if Required for Domain Adaptive Semantic Segmentationcited by 33
2021Unsupervised Domain Generalization by Learning a Bridge Across Domainscited by 33
2023Going Beyond Nouns With Vision & Language Models Using Synthetic Datacited by 33
2021Pseudo-labeling for Scalable 3D Object Detectioncited by 32
2021Contrastive Syn-to-Real Generalizationcited by 32
2021Visualizing Adapted Knowledge in Domain Transfercited by 32
2021GANs Can Play Lottery Tickets Toocited by 32
2021Self-supervised EEG Representation Learning for Automatic Sleep Stagingcited by 30
2021Confidence Calibration for Domain Generalization under Covariate Shiftcited by 27
2021Task2Sim : Towards Effective Pre-training and Transfer from Synthetic Datacited by 26
2021Domain Adaptation on Point Clouds via Geometry-Aware Implicitscited by 26
2023CF-Font: Content Fusion for Few-shot Font Generationcited by 26

50 most cited of 287 All 287 (JSON)