# Synthesizing Training Data for Object Detection in Indoor Scenes

arXiv: https://arxiv.org/abs/1702.07836 · HTML: https://mlsanity.com/paper/1702.07836/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Feb 2017.

## Built on by (most cited first, 30 of 48)

- [A Comprehensive Survey of Image Augmentation Techniques for Deep Learning](https://mlsanity.com/paper/2205.01491/) (2022)
- [Making Images Real Again: A Comprehensive Survey on Deep Image Composition](https://mlsanity.com/paper/2106.14490/) (2021)
- [MosaicOS: A Simple and Effective Use of Object-Centric Images for Long-Tailed Object Detection](https://mlsanity.com/paper/2102.08884/) (2021)
- [OPA: Object Placement Assessment Dataset](https://mlsanity.com/paper/2107.01889/) (2021)
- [Occlusion Handling in Generic Object Detection: A Review](https://mlsanity.com/paper/2101.08845/) (2021)
- [Image Compositing for Segmentation of Surgical Tools without Manual Annotations](https://mlsanity.com/paper/2102.09528/) (2021)
- [You Only Cut Once: Boosting Data Augmentation with a Single Cut](https://mlsanity.com/paper/2201.12078/) (2022)
- [Learning Object Placement via Dual-path Graph Completion](https://mlsanity.com/paper/2207.11464/) (2022)
- [A Scaling Law for Synthetic-to-Real Transfer: How Much Is Your Pre-training Effective?](https://mlsanity.com/paper/2108.11018/) (2021)
- [Multimodal Fusion of EMG and Vision for Human Grasp Intent Inference in Prosthetic Hand Control](https://mlsanity.com/paper/2104.03893/) (2021)
- [Fast Object Placement Assessment](https://mlsanity.com/paper/2205.14280/) (2022)
- [Gen2Det: Generate to Detect](https://mlsanity.com/paper/2312.04566/) (2023)
- [LLM-enhanced Scene Graph Learning for Household Rearrangement](https://mlsanity.com/paper/2408.12093/) (2024)
- [Characterizing and Improving the Robustness of Self-Supervised Learning through Background Augmentations](https://mlsanity.com/paper/2103.12719/) (2021)
- [Bayesian Active Learning for Sim-to-Real Robotic Perception](https://mlsanity.com/paper/2109.11547/) (2021)
- [Improving Object Detector Training on Synthetic Data by Starting With a Strong Baseline Methodology](https://mlsanity.com/paper/2405.19822/) (2024)
- [Solving Instance Detection from an Open-World Perspective](https://mlsanity.com/paper/2503.00359/) (2025)
- [A High-Resolution Dataset for Instance Detection with Multi-View Instance Capture](https://mlsanity.com/paper/2310.19257/) (2023)
- [MObI: Multimodal Object Inpainting Using Diffusion Models](https://mlsanity.com/paper/2501.03173/) (2025)
- [Automatically Prepare Training Data for YOLO Using Robotic In-Hand Observation and Synthesis](https://mlsanity.com/paper/2301.01441/) (2023)
- [HaDR: Applying Domain Randomization for Generating Synthetic Multimodal Dataset for Hand Instance Segmentation in Cluttered Industrial Environments](https://mlsanity.com/paper/2304.05826/) (2023)
- [DiffPop: Plausibility-Guided Object Placement Diffusion for Image Composition](https://mlsanity.com/paper/2406.07852/) (2024)
- [Robotic Waste Sorter with Agile Manipulation and Quickly Trainable Detector](https://mlsanity.com/paper/2104.01260/) (2021)
- [Beyond Classification: Knowledge Distillation using Multi-Object Impressions](https://mlsanity.com/paper/2110.14215/) (2021)
- [Scaling Law of Sim2Real Transfer Learning in Expanding Computational Materials Databases for Real-World Predictions](https://mlsanity.com/paper/2408.04042/) (2024)
- [Synthetic imagery for fuzzy object detection: A comparative study](https://mlsanity.com/paper/2410.01124/) (2024)
- [BIM Hyperreality: Data Synthesis Using BIM and Hyperrealistic Rendering for Deep Learning](https://mlsanity.com/paper/2105.04103/) (2021)
- [Automatic dataset generation for specific object detection](https://mlsanity.com/paper/2207.07867/) (2022)
- [HazardNet: Road Debris Detection by Augmentation of Synthetic Models](https://mlsanity.com/paper/2303.07547/) (2023)
- [Realistically distributing object placements in synthetic training data improves the performance of vision-based object detection models](https://mlsanity.com/paper/2305.14621/) (2023)

## Abstract (arXiv)

Detection of objects in cluttered indoor environments is one of the key enabling functionalities for service robots. The best performing object detection approaches in computer vision exploit deep Convolutional Neural Networks (CNN) to simultaneously detect and categorize the objects of interest in cluttered scenes. Training of such models typically requires large amounts of annotated training data which is time consuming and costly to obtain. In this work we explore the ability of using synthetically generated composite images for training state-of-the-art object detectors, especially for object instance detection. We superimpose 2D images of textured object models into images of real environments at variety of locations and scales. Our experiments evaluate different superimposition strategies ranging from purely image-based blending all the way to depth and semantics informed positioning of the object models into real scenes. We demonstrate the effectiveness of these object detector training strategies on two publicly available datasets, the GMU-Kitchens and the Washington RGB-D Scenes v2. As one observation, augmenting some hand-labeled training data with synthetic examples carefully composed onto scenes yields object detectors with comparable performance to using much more hand-labeled data. Broadly, this work charts new opportunities for training detectors for new objects by exploiting existing object model repositories in either a purely automatic fashion or with only a very small number of human-annotated examples.