# Making Images Real Again: A Comprehensive Survey on Deep Image Composition

arXiv: https://arxiv.org/abs/2106.14490 · HTML: https://mlsanity.com/paper/2106.14490/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- Builds on 107 of its 329 references that are in the map.
- First posted on arXiv 28 Jun 2021.

## Builds on (most cited first, 30 of 107)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Image Segmentation Using Deep Learning: A Survey](https://mlsanity.com/paper/2001.05566/) (2020)
- [Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation](https://mlsanity.com/paper/2012.07177/) (2020)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale](https://mlsanity.com/paper/1811.00982/) (2018)
- [Image Inpainting for Irregular Holes Using Partial Convolutions](https://mlsanity.com/paper/1804.07723/) (2018)
- [A Neural Algorithm of Artistic Style](https://mlsanity.com/paper/1508.06576/) (2015)
- [Semantic Scene Completion from a Single Depth Image](https://mlsanity.com/paper/1611.08974/) (2016)
- [NIMA: Neural Image Assessment](https://mlsanity.com/paper/1709.05424/) (2017)
- [Deep Learning for Image Super-resolution: A Survey](https://mlsanity.com/paper/1902.06068/) (2019)
- [Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detection](https://mlsanity.com/paper/1708.01642/) (2017)
- [Arbitrary Style Transfer with Style-Attentional Networks](https://mlsanity.com/paper/1812.02342/) (2018)
- [Deeply supervised salient object detection with short connections](https://mlsanity.com/paper/1611.04849/) (2016)
- [Deep Photo Style Transfer](https://mlsanity.com/paper/1703.07511/) (2017)
- [Learning to Predict Indoor Illumination from a Single Image](https://mlsanity.com/paper/1704.00090/) (2017)
- [Deep Learning for Single Image Super-Resolution: A Brief Review](https://mlsanity.com/paper/1808.03344/) (2018)

## Built on by (most cited first, 30 of 63)

- [Paint by Example: Exemplar-based Image Editing with Diffusion Models](https://mlsanity.com/paper/2211.13227/) (2022)
- [TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Composition](https://mlsanity.com/paper/2307.12493/) (2023)
- [Neural Gaffer: Relighting Any Object via Diffusion](https://mlsanity.com/paper/2406.07520/) (2024)
- [High-Resolution Image Harmonization via Collaborative Dual Transformations](https://mlsanity.com/paper/2109.06671/) (2021)
- [ControlCom: Controllable Image Composition using Diffusion Model](https://mlsanity.com/paper/2308.10040/) (2023)
- [3DRealCar: An In-the-wild RGB-D Car Dataset with 360-degree Views](https://mlsanity.com/paper/2406.04875/) (2024)
- [DreamEdit: Subject-driven Image Editing](https://mlsanity.com/paper/2306.12624/) (2023)
- [Neural Preset for Color Style Transfer](https://mlsanity.com/paper/2303.13511/) (2023)
- [Does FLUX Already Know How to Perform Physically Plausible Image Composition?](https://mlsanity.com/paper/2509.21278/) (2025)
- [OPA: Object Placement Assessment Dataset](https://mlsanity.com/paper/2107.01889/) (2021)
- [Semi-supervised Parametric Real-world Image Harmonization](https://mlsanity.com/paper/2303.00157/) (2023)
- [Learning Object Placement via Dual-path Graph Completion](https://mlsanity.com/paper/2207.11464/) (2022)
- [PrimeComposer: Faster Progressively Combined Diffusion for Image Composition with Attention Steering](https://mlsanity.com/paper/2403.05053/) (2024)
- [SmartMask: Context Aware High-Fidelity Mask Generation for Fine-grained Object Insertion and Layout Control](https://mlsanity.com/paper/2312.05039/) (2023)
- [DreamCom: Finetuning Text-guided Inpainting Model for Image Composition](https://mlsanity.com/paper/2309.15508/) (2023)
- [Shadow Generation for Composite Image Using Diffusion model](https://mlsanity.com/paper/2403.15234/) (2024)
- [TopNet: Transformer-based Object Placement Network for Image Compositing](https://mlsanity.com/paper/2304.03372/) (2023)
- [MMNet: Multi-Collaboration and Multi-Supervision Network for Sequential Deepfake Detection](https://mlsanity.com/paper/2307.02733/) (2023)
- [Fast Object Placement Assessment](https://mlsanity.com/paper/2205.14280/) (2022)
- [Deep Image Matting: A Comprehensive Survey](https://mlsanity.com/paper/2304.04672/) (2023)
- [Painterly Image Harmonization using Diffusion Model](https://mlsanity.com/paper/2308.02228/) (2023)
- [Deep Image Harmonization in Dual Color Spaces](https://mlsanity.com/paper/2308.02813/) (2023)
- [Scene-aware Human Pose Generation using Transformer](https://mlsanity.com/paper/2308.02177/) (2023)
- [TALE: Training-free Cross-domain Image Composition via Adaptive Latent Manipulation and Energy-guided Optimization](https://mlsanity.com/paper/2408.03637/) (2024)
- [Generative Artificial Intelligence Meets Synthetic Aperture Radar: A Survey](https://mlsanity.com/paper/2411.05027/) (2024)
- [Compose and Conquer: Diffusion-Based 3D Depth Aware Composable Image Synthesis](https://mlsanity.com/paper/2401.09048/) (2024)
- [Foreground Object Search by Distilling Composite Image Feature](https://mlsanity.com/paper/2308.04990/) (2023)
- [Repositioning the Subject within Image](https://mlsanity.com/paper/2401.16861/) (2024)
- [SAC-GAN: Structure-Aware Image Composition](https://mlsanity.com/paper/2112.06596/) (2021)
- [Deep Image Harmonization with Learnable Augmentation](https://mlsanity.com/paper/2308.00376/) (2023)

## Abstract (arXiv)

As a common image editing operation, image composition/compositing, which is also called object/subject insertion/addition/compositing, aims to combine the foreground from one image and another background image to produce a composite image. However, there are many issues that could make the composite images unrealistic. These issues can be summarized as the inconsistency between foreground and background, which includes appearance inconsistency, geometry inconsistency, and semantic inconsistency. The image composition task could be decomposed into multiple sub-tasks, in which each sub-task targets one or more issues. Specifically, object placement aims to find reasonable scale, location, and shape for the foreground. Image blending aims to address the unnatural boundary between foreground and background. Image harmonization aims to adjust the illumination statistics of foreground. Shadow (resp., reflection) generation aims to generate plausible shadow (resp., reflection) for the foreground. These sub-tasks can be executed sequentially or in parallel to acquire realistic composite images. To the best of our knowledge, there is no previous survey on image composition. In this paper, we conduct a comprehensive survey over the sub-tasks and combined task of image composition. For each one, we summarize the existing methods, available datasets, and common evaluation metrics. Datasets and codes for image composition are summarized at https://github.com/bcmi/Awesome-Object-Insertion. We have also contributed the first image composition toolbox: libcom https://github.com/bcmi/libcom, which assembles 10+ image-composition-related functions. The ultimate goal of this toolbox is to solve all image composition problems with simple `import libcom'. Based on libcom toolbox, we also develop an online image composition workbench https://libcom.ustcnewly.com.