mlsanityData to 7 Oct 2026

Meta-Sim: Learning to Generate Synthetic Datasets

arXiv 1904.11621 ↗First posted 25 Apr 2019cs.CV cs.AI cs.GRcited by 128

Not on the map. Why?

Abstract (arXiv)

Training models to high-end performance requires availability of large labeled datasets, which are expensive to get. The goal of our work is to automatically synthesize labeled datasets that are relevant for a downstream task. We propose Meta-Sim, which learns a generative model of synthetic scenes, and obtain images as well as its corresponding ground-truth via a graphics engine. We parametrize our dataset generator with a neural network, which learns to modify attributes of scene graphs obtained from probabilistic scene grammars, so as to minimize the distribution gap between its rendered outputs and target data. If the real dataset comes with a small labeled validation set, we additionally aim to optimize a meta-objective, i.e. downstream task performance. Experiments show that the proposed method can greatly improve content generation quality over a human-engineered probabilistic scene grammar, both qualitatively and quantitatively as measured by performance on a downstream task.

Key facts. Cited by 128 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 30; 2022: 27; 2023: 30; 2024: 21; 2025: 11; 2026: 9.

128 papers.

Sort:

By area:

  1. 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 720
  2. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 308
  3. 2021DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effortcited by 264
  4. 2021Fake It Till You Make It: Face analysis in the wild using synthetic data alonecited by 200
  5. 2021ATISS: Autoregressive Transformers for Indoor Scene Synthesiscited by 195
  6. 2023DiffuMask: Synthesizing Images with Pixel-level Annotations for Semantic Segmentation Using Diffusion Modelscited by 162
  7. 2021Occluded Video Instance Segmentation: A Benchmarkcited by 154
  8. 2022Ditto: Building Digital Twins of Articulated Objects from Interactioncited by 152
  9. 2021Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalizationcited by 141
  10. 2021Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Priorcited by 136
  11. 2021DriveGAN: Towards a Controllable High-Quality Neural Simulationcited by 125
  12. 2021AutoFlow: Learning a Better Training Set for Optical Flowcited by 102
38 more
  1. 2022BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotationscited by 99
  2. 2022KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradientscited by 95
  3. 2024I-Design: Personalized LLM Interior Designercited by 87
  4. 2021SceneGen: Learning to Generate Realistic Traffic Scenescited by 83
  5. 2023NeuralField-LDM: Scene Generation with Hierarchical Latent Diffusion Modelscited by 76
  6. 2021Multi-Modal 3D Object Detection in Autonomous Driving: a Surveycited by 63
  7. 2022MIME: Human-Aware 3D Scene Generationcited by 57
  8. 2023How Simulation Helps Autonomous Driving:A Survey of Sim2real, Digital Twins, and Parallel Intelligencecited by 55
  9. 2023Synthetic Datasets for Autonomous Driving: A Surveycited by 48
  10. 2023Scenario Diffusion: Controllable Driving Scenario Generation With Diffusioncited by 46
  11. 2023All for One, and One for All: UrbanSyn Dataset, the third Musketeer of Synthetic Driving Scenescited by 40
  12. 2024Solving Motion Planning Tasks with a Scalable Generative Modelcited by 40
  13. 2022On-Demand Sampling: Learning Optimally from Multiple Distributionscited by 39
  14. 2024Can Large Language Models Understand Symbolic Graphics Programs?cited by 32
  15. 20213DB: A Framework for Debugging Computer Vision Modelscited by 31
  16. 2022Behavioral Intention Prediction in Driving Scenes: A Surveycited by 30
  17. 2024Mixed Diffusion for 3D Indoor Scene Synthesiscited by 30
  18. 2023Data-driven Traffic Simulation: A Comprehensive Reviewcited by 29
  19. 2021Human-Aided Saliency Maps Improve Generalization of Deep Learningcited by 27
  20. 2022Neural-Sim: Learning to Generate Training Data with NeRFcited by 27
  21. 2021Task2Sim : Towards Effective Pre-training and Transfer from Synthetic Datacited by 26
  22. 2023Choose Your Simulator Wisely: A Review on Open-source Simulators for Autonomous Drivingcited by 26
  23. 2023GINA-3D: Learning to Generate Implicit Neural Assets in the Wildcited by 25
  24. 2024CyberDemo: Augmenting Simulated Human Demonstration for Real-World Dexterous Manipulationcited by 25
  25. 20253D Scene Generation: A Surveycited by 24
  26. 2021Unconditional Scene Graph Generationcited by 22
  27. 2021Leveraging Synthetic Data in Object Detection on Unmanned Aerial Vehiclescited by 22
  28. 2024AdvDiffuser: Generating Adversarial Safety-Critical Driving Scenarios via Guided Diffusioncited by 20
  29. 2023CabiNet: Scaling Neural Collision Detection for Object Rearrangement with Procedural Scene Generationcited by 17
  30. 2023CLIP-Layout: Style-Consistent Indoor Scene Synthesis with Semantic Furniture Embeddingcited by 16
  31. 2023Generating Driving Scenes with Diffusioncited by 16
  32. 2024A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agentscited by 16
  33. 2022Synthetic Data Supervised Salient Object Detectioncited by 15
  34. 2024Adversarial Safety-Critical Scenario Generation using Naturalistic Human Driving Priorscited by 15
  35. 2024KALIE: Fine-Tuning Vision-Language Models for Open-World Manipulation without Robot Datacited by 15
  36. 2024Blox-Net: Generative Design-for-Robot-Assembly Using VLM Supervision, Physics Simulation, and a Robot with Resetcited by 14
  37. 2022DODA: Data-oriented Sim-to-Real Domain Adaptation for 3D Semantic Segmentationcited by 13
  38. 2022Unpaired Image Translation via Vector Symbolic Architecturescited by 13

50 most cited of 128 All 128 (JSON)