# Learning to Generate Synthetic Data via Compositing

arXiv: https://arxiv.org/abs/1904.05475 · HTML: https://mlsanity.com/paper/1904.05475/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Apr 2019.

## Built on by (most cited first, 30 of 52)

- [TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Composition](https://mlsanity.com/paper/2307.12493/) (2023)
- [Mix3D: Out-of-Context Data Augmentation for 3D Scenes](https://mlsanity.com/paper/2110.02210/) (2021)
- [Making Images Real Again: A Comprehensive Survey on Deep Image Composition](https://mlsanity.com/paper/2106.14490/) (2021)
- [Does FLUX Already Know How to Perform Physically Plausible Image Composition?](https://mlsanity.com/paper/2509.21278/) (2025)
- [OPA: Object Placement Assessment Dataset](https://mlsanity.com/paper/2107.01889/) (2021)
- [Tuning-Free Image Customization with Image and Text Guidance](https://mlsanity.com/paper/2403.12658/) (2024)
- [Image Compositing for Segmentation of Surgical Tools without Manual Annotations](https://mlsanity.com/paper/2102.09528/) (2021)
- [Learning Object Placement via Dual-path Graph Completion](https://mlsanity.com/paper/2207.11464/) (2022)
- [General Intelligence Requires Rethinking Exploration](https://mlsanity.com/paper/2211.07819/) (2022)
- [PrimeComposer: Faster Progressively Combined Diffusion for Image Composition with Attention Steering](https://mlsanity.com/paper/2403.05053/) (2024)
- [Thinking Outside the BBox: Unconstrained Generative Object Compositing](https://mlsanity.com/paper/2409.04559/) (2024)
- [SmartMask: Context Aware High-Fidelity Mask Generation for Fine-grained Object Insertion and Layout Control](https://mlsanity.com/paper/2312.05039/) (2023)
- [DALL-E for Detection: Language-driven Compositional Image Synthesis for Object Detection](https://mlsanity.com/paper/2206.09592/) (2022)
- [TopNet: Transformer-based Object Placement Network for Image Compositing](https://mlsanity.com/paper/2304.03372/) (2023)
- [Fast Object Placement Assessment](https://mlsanity.com/paper/2205.14280/) (2022)
- [Deep Image Matting: A Comprehensive Survey](https://mlsanity.com/paper/2304.04672/) (2023)
- [A Comparative Review of Recent Few-Shot Object Detection Algorithms](https://mlsanity.com/paper/2111.00201/) (2021)
- [Beyond Generation: Harnessing Text to Image Models for Object Detection and Segmentation](https://mlsanity.com/paper/2309.05956/) (2023)
- [A Comprehensive Survey on Imbalanced Data Learning](https://mlsanity.com/paper/2502.08960/) (2025)
- [Scene-aware Human Pose Generation using Transformer](https://mlsanity.com/paper/2308.02177/) (2023)
- [Real-Aug: Realistic Scene Synthesis for LiDAR Augmentation in 3D Object Detection](https://mlsanity.com/paper/2305.12853/) (2023)
- [S5Mars: Semi-Supervised Learning for Mars Semantic Segmentation](https://mlsanity.com/paper/2207.01200/) (2022)
- [Foreground Object Search by Distilling Composite Image Feature](https://mlsanity.com/paper/2308.04990/) (2023)
- [Repositioning the Subject within Image](https://mlsanity.com/paper/2401.16861/) (2024)
- [Mitigating Object Dependencies: Improving Point Cloud Self-Supervised Learning through Object Exchange](https://mlsanity.com/paper/2404.07504/) (2024)
- [Longitudinal detection of new MS lesions using Deep Learning](https://mlsanity.com/paper/2206.08272/) (2022)
- [Imagining the Unseen: Generative Location Modeling for Object Placement](https://mlsanity.com/paper/2410.13564/) (2024)
- [Cut-and-Splat: Leveraging Gaussian Splatting for Synthetic Data Generation](https://mlsanity.com/paper/2504.08473/) (2025)
- [Road images augmentation with synthetic traffic signs using neural networks](https://mlsanity.com/paper/2101.04927/) (2021)
- [DiffPop: Plausibility-Guided Object Placement Diffusion for Image Composition](https://mlsanity.com/paper/2406.07852/) (2024)

## Abstract (arXiv)

We present a task-aware approach to synthetic data generation. Our framework employs a trainable synthesizer network that is optimized to produce meaningful training samples by assessing the strengths and weaknesses of a `target' network. The synthesizer and target networks are trained in an adversarial manner wherein each network is updated with a goal to outdo the other. Additionally, we ensure the synthesizer generates realistic data by pairing it with a discriminator trained on real-world images. Further, to make the target classifier invariant to blending artefacts, we introduce these artefacts to background regions of the training images so the target does not over-fit to them. We demonstrate the efficacy of our approach by applying it to different target networks including a classification network on AffNIST, and two object detection networks (SSD, Faster-RCNN) on different datasets. On the AffNIST benchmark, our approach is able to surpass the baseline results with just half the training examples. On the VOC person detection benchmark, we show improvements of up to 2.7% as a result of our data augmentation. Similarly on the GMU detection benchmark, we report a performance boost of 3.5% in mAP over the baseline method, outperforming the previous state of the art approaches by up to 7.5% on specific categories.