# Segment Anything Model Meets Image Harmonization

arXiv: https://arxiv.org/abs/2312.12729 · HTML: https://mlsanity.com/paper/2312.12729/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 24 references that are in the map.
- First posted on arXiv 20 Dec 2023.

## Builds on (most cited first, 19 of 19)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [AnyDoor: Zero-shot Object-level Image Customization](https://mlsanity.com/paper/2307.09481/) (2023)
- [DoveNet: Deep Image Harmonization via Domain Verification](https://mlsanity.com/paper/1911.13239/) (2019)
- [Deep Image Harmonization](https://mlsanity.com/paper/1703.00069/) (2017)
- [Improving the Harmony of the Composite Image by Spatial-Separated Attention Module](https://mlsanity.com/paper/1907.06406/) (2019)
- [Region-aware Adaptive Instance Normalization for Image Harmonization](https://mlsanity.com/paper/2106.02853/) (2021)
- [Harmonizer: Learning to Perform White-Box Image and Video Harmonization](https://mlsanity.com/paper/2207.01322/) (2022)
- [SSH: A Self-Supervised Framework for Image Harmonization](https://mlsanity.com/paper/2108.06805/) (2021)
- [High-Resolution Image Harmonization via Collaborative Dual Transformations](https://mlsanity.com/paper/2109.06671/) (2021)
- [BargainNet: Background-Guided Domain Translation for Image Harmonization](https://mlsanity.com/paper/2009.09169/) (2020)
- [DCCF: Deep Comprehensible Color Filter Learning Framework for High-Resolution Image Harmonization](https://mlsanity.com/paper/2207.04788/) (2022)
- [DiffUTE: Universal Text Editing Diffusion Model](https://mlsanity.com/paper/2305.10825/) (2023)
- [Spatial-Separated Curve Rendering Network for Efficient and High-Resolution Image Harmonization](https://mlsanity.com/paper/2109.05750/) (2021)
- [SCS-Co: Self-Consistent Style Contrastive Learning for Image Harmonization](https://mlsanity.com/paper/2204.13962/) (2022)
- [Dense Pixel-to-Pixel Harmonization via Continuous Image Representation](https://mlsanity.com/paper/2303.01681/) (2023)
- [Hierarchical Dynamic Image Harmonization](https://mlsanity.com/paper/2211.08639/) (2022)
- [LEMaRT: Label-Efficient Masked Region Transform for Image Harmonization](https://mlsanity.com/paper/2304.13166/) (2023)
- [Learning Global-aware Kernel for Image Harmonization](https://mlsanity.com/paper/2305.11676/) (2023)
- [Deep Image Harmonization with Globally Guided Feature Transformation and Relation Distillation](https://mlsanity.com/paper/2308.00356/) (2023)

## Built on by (most cited first, 1 of 1)

- [SAMReg: SAM-enabled Image Registration with ROI-based Correspondence](https://mlsanity.com/paper/2410.14083/) (2024)

## Abstract (arXiv)

Image harmonization is a crucial technique in image composition that aims to seamlessly match the background by adjusting the foreground of composite images. Current methods adopt either global-level or pixel-level feature matching. Global-level feature matching ignores the proximity prior, treating foreground and background as separate entities. On the other hand, pixel-level feature matching loses contextual information. Therefore, it is necessary to use the information from semantic maps that describe different objects to guide harmonization. In this paper, we propose Semantic-guided Region-aware Instance Normalization (SRIN) that can utilize the semantic segmentation maps output by a pre-trained Segment Anything Model (SAM) to guide the visual consistency learning of foreground and background features. Abundant experiments demonstrate the superiority of our method for image harmonization over state-of-the-art methods.