# Context-Aware Synthesis and Placement of Object Instances

arXiv: https://arxiv.org/abs/1812.02350 · HTML: https://mlsanity.com/paper/1812.02350/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Dec 2018.

## Built on by (most cited first, 30 of 41)

- [GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Driving](https://mlsanity.com/paper/2101.06543/) (2021)
- [Perspective Fields for Single Image Camera Calibration](https://mlsanity.com/paper/2212.03239/) (2022)
- [Making Images Real Again: A Comprehensive Survey on Deep Image Composition](https://mlsanity.com/paper/2106.14490/) (2021)
- [OPA: Object Placement Assessment Dataset](https://mlsanity.com/paper/2107.01889/) (2021)
- [Learning Object Placement via Dual-path Graph Completion](https://mlsanity.com/paper/2207.11464/) (2022)
- [Thinking Outside the BBox: Unconstrained Generative Object Compositing](https://mlsanity.com/paper/2409.04559/) (2024)
- [SmartMask: Context Aware High-Fidelity Mask Generation for Fine-grained Object Insertion and Layout Control](https://mlsanity.com/paper/2312.05039/) (2023)
- [DreamCom: Finetuning Text-guided Inpainting Model for Image Composition](https://mlsanity.com/paper/2309.15508/) (2023)
- [Magic Insert: Style-Aware Drag-and-Drop](https://mlsanity.com/paper/2407.02489/) (2024)
- [TopNet: Transformer-based Object Placement Network for Image Compositing](https://mlsanity.com/paper/2304.03372/) (2023)
- [LayoutDETR: Detection Transformer Is a Good Multimodal Layout Designer](https://mlsanity.com/paper/2212.09877/) (2022)
- [Fast Object Placement Assessment](https://mlsanity.com/paper/2205.14280/) (2022)
- [Click to Move: Controlling Video Generation with Sparse Motion](https://mlsanity.com/paper/2108.08815/) (2021)
- [Beyond Generation: Harnessing Text to Image Models for Object Detection and Segmentation](https://mlsanity.com/paper/2309.05956/) (2023)
- [ISF-GAN: An Implicit Style Function for High-Resolution Image-to-Image Translation](https://mlsanity.com/paper/2109.12492/) (2021)
- [Scene-aware Human Pose Generation using Transformer](https://mlsanity.com/paper/2308.02177/) (2023)
- [SemIE: Semantically-aware Image Extrapolation](https://mlsanity.com/paper/2108.13702/) (2021)
- [Foreground Object Search by Distilling Composite Image Feature](https://mlsanity.com/paper/2308.04990/) (2023)
- [GaussiGAN: Controllable Image Synthesis with 3D Gaussians from Unposed Silhouettes](https://mlsanity.com/paper/2106.13215/) (2021)
- [SAC-GAN: Structure-Aware Image Composition](https://mlsanity.com/paper/2112.06596/) (2021)
- [GALA: Toward Geometry-and-Lighting-Aware Object Search for Compositing](https://mlsanity.com/paper/2204.00125/) (2022)
- [Imagining the Unseen: Generative Location Modeling for Object Placement](https://mlsanity.com/paper/2410.13564/) (2024)
- [MonoPlace3D: Learning 3D-Aware Object Placement for 3D Monocular Detection](https://mlsanity.com/paper/2504.06801/) (2025)
- [Road images augmentation with synthetic traffic signs using neural networks](https://mlsanity.com/paper/2101.04927/) (2021)
- [LayoutBERT: Masked Language Layout Model for Object Insertion](https://mlsanity.com/paper/2205.00347/) (2022)
- [Animating Street View](https://mlsanity.com/paper/2310.08534/) (2023)
- [Addressing Limitations of State-Aware Imitation Learning for Autonomous Driving](https://mlsanity.com/paper/2310.20650/) (2023)
- [DiffPop: Plausibility-Guided Object Placement Diffusion for Image Composition](https://mlsanity.com/paper/2406.07852/) (2024)
- [Spatial Transformation for Image Composition via Correspondence Learning](https://mlsanity.com/paper/2207.02398/) (2022)
- [Scene-Aware Location Modeling for Data Augmentation in Automotive Object Detection](https://mlsanity.com/paper/2504.17076/) (2025)

## Abstract (arXiv)

Learning to insert an object instance into an image in a semantically coherent manner is a challenging and interesting problem. Solving it requires (a) determining a location to place an object in the scene and (b) determining its appearance at the location. Such an object insertion model can potentially facilitate numerous image editing and scene parsing applications. In this paper, we propose an end-to-end trainable neural network for the task of inserting an object instance mask of a specified class into the semantic label map of an image. Our network consists of two generative modules where one determines where the inserted object mask should be (i.e., location and scale) and the other determines what the object mask shape (and pose) should look like. The two modules are connected together via a spatial transformation network and jointly trained. We devise a learning procedure that leverage both supervised and unsupervised data and show our model can insert an object at diverse locations with various appearances. We conduct extensive experimental validations with comparisons to strong baselines to verify the effectiveness of the proposed network.