mlsanityData to 7 Oct 2026

Photorealistic Image Synthesis for Object Instance Detection

arXiv 1902.03334 ↗First posted 9 Feb 2019cs.CV cs.AI cs.ROcited by 66

Not on the map. Why?

Abstract (arXiv)

We present an approach to synthesize highly photorealistic images of 3D object models, which we use to train a convolutional neural network for detecting the objects in real images. The proposed approach has three key ingredients: (1) 3D object models are rendered in 3D models of complete scenes with realistic materials and lighting, (2) plausible geometric configuration of objects and cameras in a scene is generated using physics simulations, and (3) high photorealism of the synthesized images achieved by physically based rendering. When trained on images synthesized by the proposed approach, the Faster R-CNN object detector achieves a 24% absolute improvement of [email protected] on Rutgers APC and 11% on LineMod-Occluded datasets, compared to a baseline where the training images are synthesized by rendering object models on top of random photographs. This work is a step towards being able to effectively train object detectors without capturing or annotating any real images. A dataset of 600K synthetic images with ground truth annotations for various computer vision tasks will be released on the project website: thodan.github.io/objectsynth.

Key facts. Cited by 66 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 20; 2023: 13; 2024: 12; 2025: 7; 2026: 2.

66 papers.

Sort:

By area:

  1. 2021Fake It Till You Make It: Face analysis in the wild using synthetic data alonecited by 200
  2. 2024HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videoscited by 122
  3. 2023FoundPose: Unseen Object Pose Estimation with Foundation Featurescited by 109
  4. 2023BOP Challenge 2022 on Detection, Segmentation and Pose Estimation of Specific Rigid Objectscited by 70
  5. 2021Multi-Modal 3D Object Detection in Autonomous Driving: a Surveycited by 63
  6. 2024BOP Challenge 2023 on Detection, Segmentation and Pose Estimation of Seen and Unseen Rigid Objectscited by 60
  7. 2023Portrait4D: Learning One-Shot 4D Head Avatar Synthesis using Synthetic Datacited by 55
  8. 2022X-Paste: Revisiting Scalable Copy-Paste for Instance Segmentation using CLIP and StableDiffusioncited by 49
  9. 2024Introducing HOT3D: An Egocentric Dataset for 3D Hand and Object Trackingcited by 46
  10. 2022Occlusion-Aware Self-Supervised Monocular 6D Object Pose Estimationcited by 34
  11. 2022Perspective Flow Aggregation for Data-Limited 6D Object Pose Estimationcited by 32
  12. 2022Neural-Sim: Learning to Generate Training Data with NeRFcited by 27
38 more
  1. 2021ZePHyR: Zero-shot Pose Hypothesis Ratingcited by 26
  2. 2023Challenges for Monocular 6D Object Pose Estimation in Roboticscited by 21
  3. 2021DemoGrasp: Few-Shot Learning for Robotic Grasping with Human Demonstrationcited by 20
  4. 2022DALL-E for Detection: Language-driven Compositional Image Synthesis for Object Detectioncited by 17
  5. 2023Instance Tracking in 3D Scenes from Egocentric Videoscited by 17
  6. 2021CloudAAE: Learning 6D Object Pose Regression with On-line Data Synthesis on Point Cloudscited by 15
  7. 2021DSC-PoseNet: Learning 6DoF Object Pose Estimation via Dual-scale Consistencycited by 14
  8. 2021A Scaling Law for Synthetic-to-Real Transfer: How Much Is Your Pre-training Effective?cited by 14
  9. 2024PEGASUS: Physically Enhanced Gaussian Splatting Simulation System for 6DoF Object Pose Dataset Generationcited by 13
  10. 2022COPE: End-to-end trainable Constant Runtime Object Pose Estimationcited by 10
  11. 2023Beyond Generation: Harnessing Text to Image Models for Object Detection and Segmentationcited by 10
  12. 2022Analysis of Training Object Detection Models with Synthetic Datacited by 9
  13. 2021Bayesian Active Learning for Sim-to-Real Robotic Perceptioncited by 8
  14. 2022OSSID: Online Self-Supervised Instance Detection by (and for) Pose Estimationcited by 8
  15. 2022Dataset of Industrial Metal Objectscited by 8
  16. 2023TransPose: 6D Object Pose Estimation with Geometry-Aware Transformercited by 7
  17. 2021Physics-based Differentiable Depth Sensor Simulationcited by 6
  18. 2022Ambiguity-Aware Multi-Object Pose Optimization for Visually-Assisted Robot Manipulationcited by 6
  19. 2025Solving Instance Detection from an Open-World Perspectivecited by 6
  20. 2021Synthetic training data generation for deep learning based quality inspectioncited by 5
  21. 2022Mesh-Tension Driven Expression-Based Wrinkles for Synthetic Facescited by 5
  22. 2022Photo-realistic Neural Domain Randomizationcited by 5
  23. 2023Generative AI in Agriculture: Creating Image Datasets Using DALL.E's Advanced Large Language Model Capabilitiescited by 5
  24. 2024RaSim: A Range-aware High-fidelity RGB-D Data Simulation Pipeline for Real-world Applicationscited by 5
  25. 2022EM-Paste: EM-guided Cut-Paste with DALL-E Augmentation for Image-level Weakly Supervised Instance Segmentationcited by 4
  26. 2023A High-Resolution Dataset for Instance Detection with Multi-View Instance Capturecited by 4
  27. 2023Synthetic Data Generation for Bridging Sim2Real Gap in a Production Environmentcited by 4
  28. 2021SynPick: A Dataset for Dynamic Bin Picking Scene Understandingcited by 3
  29. 2022Combining Local and Global Pose Estimation for Precise Tracking of Similar Objectscited by 3
  30. 2024Synthetica: Large Scale Synthetic Data for Robot Perceptioncited by 3
  31. 2025HccePose(BF): Predicting Front & Back Surfaces to Construct Ultra-Dense 2D-3D Correspondences for Pose Estimationcited by 3
  32. 2021MBAPose: Mask and Bounding-Box Aware Pose Estimation of Surgical Instruments with Photorealistic Domain Randomizationcited by 2
  33. 2022CenDerNet: Center and Curvature Representations for Render-and-Compare 6D Pose Estimationcited by 2
  34. 2022Synthetic Image Data for Deep Learningcited by 2
  35. 2023Manifold-Aware Self-Training for Unsupervised Domain Adaptation on Regressing 6D Object Posecited by 2
  36. 2024AccessLens: Auto-detecting Inaccessibility of Everyday Objectscited by 2
  37. 2024Shape-biased Texture Agnostic Representations for Improved Textureless and Metallic Object Detection and 6D Pose Estimationcited by 2
  38. 2024NeuSurfEmb: A Complete Pipeline for Dense Correspondence-based 6D Object Pose Estimation without CAD Modelscited by 2

50 most cited of 66 All 66 (JSON)