# Improving the Harmony of the Composite Image by Spatial-Separated Attention Module

arXiv: https://arxiv.org/abs/1907.06406 · HTML: https://mlsanity.com/paper/1907.06406/

## Key facts

- Cited by 73 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jul 2019.

## Built on by (most cited first, 30 of 73)

- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Paint by Example: Exemplar-based Image Editing with Diffusion Models](https://mlsanity.com/paper/2211.13227/) (2022)
- [TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Composition](https://mlsanity.com/paper/2307.12493/) (2023)
- [Region-aware Adaptive Instance Normalization for Image Harmonization](https://mlsanity.com/paper/2106.02853/) (2021)
- [Making Images Real Again: A Comprehensive Survey on Deep Image Composition](https://mlsanity.com/paper/2106.14490/) (2021)
- [MagicStick: Controllable Video Editing via Control Handle Transformations](https://mlsanity.com/paper/2312.03047/) (2023)
- [Harmonizer: Learning to Perform White-Box Image and Video Harmonization](https://mlsanity.com/paper/2207.01322/) (2022)
- [SSH: A Self-Supervised Framework for Image Harmonization](https://mlsanity.com/paper/2108.06805/) (2021)
- [Unified Multi-Modal Latent Diffusion for Joint Subject and Text Conditional Image Generation](https://mlsanity.com/paper/2303.09319/) (2023)
- [High-Resolution Image Harmonization via Collaborative Dual Transformations](https://mlsanity.com/paper/2109.06671/) (2021)
- [TransFill: Reference-guided Image Inpainting by Merging Multiple Color and Spatial Transformations](https://mlsanity.com/paper/2103.15982/) (2021)
- [DCCF: Deep Comprehensible Color Filter Learning Framework for High-Resolution Image Harmonization](https://mlsanity.com/paper/2207.04788/) (2022)
- [ControlCom: Controllable Image Composition using Diffusion Model](https://mlsanity.com/paper/2308.10040/) (2023)
- [Neural Preset for Color Style Transfer](https://mlsanity.com/paper/2303.13511/) (2023)
- [High-Resolution Document Shadow Removal via A Large-Scale Real-World Dataset and A Frequency-Aware Shadow Erasing Net](https://mlsanity.com/paper/2308.14221/) (2023)
- [Shadow Generation for Composite Image in Real-world Scenes](https://mlsanity.com/paper/2104.10338/) (2021)
- [Does FLUX Already Know How to Perform Physically Plausible Image Composition?](https://mlsanity.com/paper/2509.21278/) (2025)
- [Spatial-Separated Curve Rendering Network for Efficient and High-Resolution Image Harmonization](https://mlsanity.com/paper/2109.05750/) (2021)
- [Tuning-Free Image Customization with Image and Text Guidance](https://mlsanity.com/paper/2403.12658/) (2024)
- [SCS-Co: Self-Consistent Style Contrastive Learning for Image Harmonization](https://mlsanity.com/paper/2204.13962/) (2022)
- [Paste, Inpaint and Harmonize via Denoising: Subject-Driven Image Editing with Pre-Trained Diffusion Model](https://mlsanity.com/paper/2306.07596/) (2023)
- [PrimeComposer: Faster Progressively Combined Diffusion for Image Composition with Attention Steering](https://mlsanity.com/paper/2403.05053/) (2024)
- [Dense Pixel-to-Pixel Harmonization via Continuous Image Representation](https://mlsanity.com/paper/2303.01681/) (2023)
- [DreamCom: Finetuning Text-guided Inpainting Model for Image Composition](https://mlsanity.com/paper/2309.15508/) (2023)
- [Shadow Generation for Composite Image Using Diffusion model](https://mlsanity.com/paper/2403.15234/) (2024)
- [Cross-domain Compositing with Pretrained Diffusion Models](https://mlsanity.com/paper/2302.10167/) (2023)
- [Fast Object Placement Assessment](https://mlsanity.com/paper/2205.14280/) (2022)
- [Hierarchical Dynamic Image Harmonization](https://mlsanity.com/paper/2211.08639/) (2022)
- [LEMaRT: Label-Efficient Masked Region Transform for Image Harmonization](https://mlsanity.com/paper/2304.13166/) (2023)
- [Painterly Image Harmonization using Diffusion Model](https://mlsanity.com/paper/2308.02228/) (2023)

## Abstract (arXiv)

Image composition is one of the most important applications in image processing. However, the inharmonious appearance between the spliced region and background degrade the quality of the image. Thus, we address the problem of Image Harmonization: Given a spliced image and the mask of the spliced region, we try to harmonize the "style" of the pasted region with the background (non-spliced region). Previous approaches have been focusing on learning directly by the neural network. In this work, we start from an empirical observation: the differences can only be found in the spliced region between the spliced image and the harmonized result while they share the same semantic information and the appearance in the non-spliced region. Thus, in order to learn the feature map in the masked region and the others individually, we propose a novel attention module named Spatial-Separated Attention Module (S2AM). Furthermore, we design a novel image harmonization framework by inserting the S2AM in the coarser low-level features of the Unet structure in two different ways. Besides image harmonization, we make a big step for harmonizing the composite image without the specific mask under previous observation. The experiments show that the proposed S2AM performs better than other state-of-the-art attention modules in our task. Moreover, we demonstrate the advantages of our model against other state-of-the-art image harmonization methods via criteria from multiple points of view. Code is available at https://github.com/vinthony/s2am