# TopNet: Transformer-based Object Placement Network for Image Compositing

arXiv: https://arxiv.org/abs/2304.03372 · HTML: https://mlsanity.com/paper/2304.03372/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 30 references that are in the map.
- First posted on arXiv 6 Apr 2023.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [CenterMask : Real-Time Anchor-Free Instance Segmentation](https://mlsanity.com/paper/1911.06667/) (2019)
- [ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositing](https://mlsanity.com/paper/1803.01837/) (2018)
- [Putting Humans in a Scene: Learning Affordance in 3D Indoor Environments](https://mlsanity.com/paper/1903.05690/) (2019)
- [Making Images Real Again: A Comprehensive Survey on Deep Image Composition](https://mlsanity.com/paper/2106.14490/) (2021)
- [Learning to Generate Synthetic Data via Compositing](https://mlsanity.com/paper/1904.05475/) (2019)
- [Context-Aware Synthesis and Placement of Object Instances](https://mlsanity.com/paper/1812.02350/) (2018)
- [OPA: Object Placement Assessment Dataset](https://mlsanity.com/paper/2107.01889/) (2021)
- [Where and Who? Automatic Semantic-Aware Person Composition](https://mlsanity.com/paper/1706.01021/) (2017)
- [Deep Image Compositing](https://mlsanity.com/paper/2011.02146/) (2020)
- [GALA: Toward Geometry-and-Lighting-Aware Object Search for Compositing](https://mlsanity.com/paper/2204.00125/) (2022)

## Built on by (most cited first, 16 of 16)

- [Thinking Outside the BBox: Unconstrained Generative Object Compositing](https://mlsanity.com/paper/2409.04559/) (2024)
- [SmartMask: Context Aware High-Fidelity Mask Generation for Fine-grained Object Insertion and Layout Control](https://mlsanity.com/paper/2312.05039/) (2023)
- [Text2Place: Affordance-aware Text Guided Human Placement](https://mlsanity.com/paper/2407.15446/) (2024)
- [PlaceIt3D: Language-Guided Object Placement in Real 3D Scenes](https://mlsanity.com/paper/2505.05288/) (2025)
- [Imagining the Unseen: Generative Location Modeling for Object Placement](https://mlsanity.com/paper/2410.13564/) (2024)
- [DiffPop: Plausibility-Guided Object Placement Diffusion for Image Composition](https://mlsanity.com/paper/2406.07852/) (2024)
- [BOOTPLACE: Bootstrapped Object Placement with Detection Transformers](https://mlsanity.com/paper/2503.21991/) (2025)
- [Object Placement for Anything](https://mlsanity.com/paper/2504.12029/) (2025)
- [AnimateScene: Camera-controllable Animation in Any Scene](https://mlsanity.com/paper/2508.05982/) (2025)
- [Scene-Aware Location Modeling for Data Augmentation in Automotive Object Detection](https://mlsanity.com/paper/2504.17076/) (2025)
- [Exploring Mutual Cross-Modal Attention for Context-Aware Human Affordance Generation](https://mlsanity.com/paper/2502.13637/) (2025)
- [GraPLUS: Graph-based Placement Using Semantics for Image Composition](https://mlsanity.com/paper/2503.15761/) (2025)
- [Weakly Supervised Object Segmentation by Background Conditional Divergence](https://mlsanity.com/paper/2506.22505/) (2025)
- [Beyond Realism: Learning the Art of Expressive Composition with StickerNet](https://mlsanity.com/paper/2511.20957/) (2025)
- [HiddenObjects: Scalable Diffusion-Distilled Spatial Priors for Object Placement](https://mlsanity.com/paper/2604.10675/) (2026)
- [Learning to Understand Body Language from Flight through Robust 3D Avatar Placing](https://mlsanity.com/paper/2607.27865/) (2026)

## Abstract (arXiv)

We investigate the problem of automatically placing an object into a background image for image compositing. Given a background image and a segmented object, the goal is to train a model to predict plausible placements (location and scale) of the object for compositing. The quality of the composite image highly depends on the predicted location/scale. Existing works either generate candidate bounding boxes or apply sliding-window search using global representations from background and object images, which fail to model local information in background images. However, local clues in background images are important to determine the compatibility of placing the objects with certain locations/scales. In this paper, we propose to learn the correlation between object features and all local background features with a transformer module so that detailed information can be provided on all possible location/scale configurations. A sparse contrastive loss is further proposed to train our model with sparse supervision. Our new formulation generates a 3D heatmap indicating the plausibility of all location/scale combinations in one network forward pass, which is over 10 times faster than the previous sliding-window method. It also supports interactive search when users provide a pre-defined location or scale. The proposed method can be trained with explicit annotation or in a self-supervised manner using an off-the-shelf inpainting model, and it outperforms state-of-the-art methods significantly. The user study shows that the trained model generalizes well to real-world images with diverse challenging scenes and object categories.