mlsanityData to 7 Oct 2026

Playing for Data: Ground Truth from Computer Games

arXiv 1608.02192 ↗First posted 7 Aug 2016cs.CVcited by 816

Not on the map. Why?

Cited by 25 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent progress in computer vision has been driven by high-capacity models trained on large datasets. Unfortunately, creating large datasets with pixel-level labels has been extremely costly due to the amount of human effort required. In this paper, we present an approach to rapidly creating pixel-accurate semantic label maps for images extracted from modern computer games. Although the source code and the internal operation of commercial games are inaccessible, we show that associations between image patches can be reconstructed from the communication between the game and the graphics hardware. This enables rapid propagation of semantic labels within and across images synthesized by the game, with no access to the source code or the content. We validate the presented approach by producing dense pixel-level semantic annotations for 25 thousand images synthesized by a photorealistic open-world computer game. Experiments on semantic segmentation datasets show that using the acquired data to supplement real-world images significantly increases accuracy and that the acquired data enables reducing the amount of hand-labeled real-world data: models trained with game data and just 1/3 of the CamVid training set outperform models trained on the complete CamVid training set.

Key facts. Cited by 816 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Aug 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 177; 2022: 162; 2023: 178; 2024: 149; 2025: 93; 2026: 56.

816 papers.

Sort:

By area:

  1. 2024Depth Anything V2cited by 1,385
  2. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 762
  3. 2021Domain Generalization: A Surveycited by 760
  4. 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 720
  5. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
  6. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 570
  7. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
  8. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  9. 2022Kubric: A scalable dataset generatorcited by 373
  10. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 372
  11. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  12. 2022Is synthetic data from generative models ready for image recognition?cited by 320
38 more
  1. 2021DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentationcited by 299
  2. 2021Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentationcited by 268
  3. 2021SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retrainingcited by 235
  4. 2021RobustNet: Improving Domain Generalization in Urban-Scene Segmentation via Instance Selective Whiteningcited by 213
  5. 2021Fake It Till You Make It: Face analysis in the wild using synthetic data alonecited by 200
  6. 2022Fake it till you make it: Learning transferable representations from synthetic ImageNet clonescited by 171
  7. 2021Source-Free Domain Adaptation for Semantic Segmentationcited by 168
  8. 2021FSDR: Frequency Space Domain Randomization for Domain Generalizationcited by 163
  9. 2022HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentationcited by 163
  10. 2021Model Adaptation: Historical Contrastive Learning for Unsupervised Domain Adaptation without Source Datacited by 152
  11. 2021Self-supervised Augmentation Consistency for Adapting Semantic Segmentationcited by 149
  12. 2023DynamicStereo: Consistent Dynamic Depth from Stereo Videoscited by 147
  13. 2022SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptationcited by 146
  14. 2023Infinite Photorealistic Worlds using Procedural Generationcited by 142
  15. 2021OpenGAN: Open-Set Recognition via Open Data Generationcited by 135
  16. 2024Infinigen Indoors: Photorealistic Indoor Scenes using Procedural Generationcited by 129
  17. 2021Semi-Supervised Semantic Segmentation with Pixel-Level Contrastive Learning from a Class-wise Memory Bankcited by 125
  18. 2021MSeg: A Composite Dataset for Multi-domain Semantic Segmentationcited by 122
  19. 2022Uncertainty Modeling for Out-of-Distribution Generalizationcited by 122
  20. 2021Learning to Track with Object Permanencecited by 117
  21. 2021Predicting with Confidence on Unseen Distributionscited by 116
  22. 2023TTN: A Domain-Shift Aware Batch Normalization in Test-Time Adaptationcited by 101
  23. 2022BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotationscited by 99
  24. 2022Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectivescited by 98
  25. 2024DriveArena: A Closed-loop Generative Simulation Platform for Autonomous Drivingcited by 98
  26. 2022High-Quality Entity Segmentationcited by 93
  27. 2023Improving Multimodal Datasets with Image Captioningcited by 93
  28. 2022Style-Hallucinated Dual Consistency Learning for Domain Generalized Semantic Segmentationcited by 88
  29. 2022SePiCo: Semantic-Guided Pixel Contrast for Domain Adaptive Semantic Segmentationcited by 87
  30. 2023Language Conditioned Traffic Generationcited by 86
  31. 2021Extending the WILDS Benchmark for Unsupervised Adaptationcited by 85
  32. 2023Stronger, Fewer, & Superior: Harnessing Vision Foundation Models for Domain Generalized Semantic Segmentationcited by 82
  33. 2021Generalize then Adapt: Source-Free Domain Adaptive Semantic Segmentationcited by 81
  34. 2021Monocular Quasi-Dense 3D Object Trackingcited by 80
  35. 2021DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime Semantic Segmentationcited by 80
  36. 2021Domain Adaptive Semantic Segmentation with Self-Supervised Depth Estimationcited by 80
  37. 2021PixMatch: Unsupervised Domain Adaptation via Pixelwise Consistency Trainingcited by 80
  38. 2021Context-Aware Mixup for Domain Adaptive Semantic Segmentationcited by 79

50 most cited of 816 All 816 (JSON)