# FRIH: Fine-grained Region-aware Image Harmonization

arXiv: https://arxiv.org/abs/2205.06448 · HTML: https://mlsanity.com/paper/2205.06448/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 45 references that are in the map.
- First posted on arXiv 13 May 2022.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [Conditional Image Generation with PixelCNN Decoders](https://mlsanity.com/paper/1606.05328/) (2016)
- [Hybrid Task Cascade for Instance Segmentation](https://mlsanity.com/paper/1901.07518/) (2019)
- [DRAW: A Recurrent Neural Network For Image Generation](https://mlsanity.com/paper/1502.04623/) (2015)
- [DoveNet: Deep Image Harmonization via Domain Verification](https://mlsanity.com/paper/1911.13239/) (2019)
- [Deep Image Harmonization](https://mlsanity.com/paper/1703.00069/) (2017)
- [Chained-Tracker: Chaining Paired Attentive Regression Results for End-to-End Joint Multiple-Object Detection and Tracking](https://mlsanity.com/paper/2007.14557/) (2020)
- [SemanticAdv: Generating Adversarial Examples via Attribute-conditional Image Editing](https://mlsanity.com/paper/1906.07927/) (2019)
- [Improving the Harmony of the Composite Image by Spatial-Separated Attention Module](https://mlsanity.com/paper/1907.06406/) (2019)
- [Region-aware Adaptive Instance Normalization for Image Harmonization](https://mlsanity.com/paper/2106.02853/) (2021)
- [SSH: A Self-Supervised Framework for Image Harmonization](https://mlsanity.com/paper/2108.06805/) (2021)
- [Foreground-aware Semantic Representations for Image Harmonization](https://mlsanity.com/paper/2006.00809/) (2020)
- [Learning a Discriminative Model for the Perception of Realism in Composite Images](https://mlsanity.com/paper/1510.00477/) (2015)
- [BargainNet: Background-Guided Domain Translation for Image Harmonization](https://mlsanity.com/paper/2009.09169/) (2020)
- [Cascade EF-GAN: Progressive Facial Expression Editing with Local Focuses](https://mlsanity.com/paper/2003.05905/) (2020)
- [Sequential Attention GAN for Interactive Image Editing](https://mlsanity.com/paper/1812.08352/) (2018)
- [Spatial-Separated Curve Rendering Network for Efficient and High-Resolution Image Harmonization](https://mlsanity.com/paper/2109.05750/) (2021)
- [Adversarial Image Composition with Auxiliary Illumination](https://mlsanity.com/paper/2009.08255/) (2020)
- [Progressive Image Inpainting with Full-Resolution Residual Network](https://mlsanity.com/paper/1907.10478/) (2019)
- [Composition-Aware Image Aesthetics Assessment](https://mlsanity.com/paper/1907.10801/) (2019)
- [CU-Net: Cascaded U-Net with Loss Weighted Sampling for Brain Tumor Segmentation](https://mlsanity.com/paper/1907.07677/) (2019)

## Built on by (most cited first, 9 of 9)

- [Dual-Interrelated Diffusion Model for Few-Shot Anomaly Image Generation](https://mlsanity.com/paper/2408.13509/) (2024)
- [UniCombine: Unified Multi-Conditional Combination with Diffusion Transformer](https://mlsanity.com/paper/2503.09277/) (2025)
- [DreamFuse: Adaptive Image Fusion with Diffusion Transformer](https://mlsanity.com/paper/2504.08291/) (2025)
- [LEMaRT: Label-Efficient Masked Region Transform for Image Harmonization](https://mlsanity.com/paper/2304.13166/) (2023)
- [Deep Image Harmonization in Dual Color Spaces](https://mlsanity.com/paper/2308.02813/) (2023)
- [Deep Image Harmonization with Globally Guided Feature Transformation and Relation Distillation](https://mlsanity.com/paper/2308.00356/) (2023)
- [Painterly Image Harmonization by Learning from Painterly Objects](https://mlsanity.com/paper/2312.10263/) (2023)
- [Progressive Painterly Image Harmonization from Low-level Styles to High-level Styles](https://mlsanity.com/paper/2312.10264/) (2023)
- [EFDiT: Efficient Fine-grained Image Generation Using Diffusion Transformer Models](https://mlsanity.com/paper/2512.05152/) (2025)

## Abstract (arXiv)

Image harmonization aims to generate a more realistic appearance of foreground and background for a composite image. Existing methods perform the same harmonization process for the whole foreground. However, the implanted foreground always contains different appearance patterns. All the existing solutions ignore the difference of each color block and losing some specific details. Therefore, we propose a novel global-local two stages framework for Fine-grained Region-aware Image Harmonization (FRIH), which is trained end-to-end. In the first stage, the whole input foreground mask is used to make a global coarse-grained harmonization. In the second stage, we adaptively cluster the input foreground mask into several submasks by the corresponding pixel RGB values in the composite image. Each submask and the coarsely adjusted image are concatenated respectively and fed into a lightweight cascaded module, adjusting the global harmonization performance according to the region-aware local feature. Moreover, we further designed a fusion prediction module by fusing features from all the cascaded decoder layers together to generate the final result, which could utilize the different degrees of harmonization results comprehensively. Without bells and whistles, our FRIH algorithm achieves the best performance on iHarmony4 dataset (PSNR is 38.19 dB) with a lightweight model. The parameters for our model are only 11.98 M, far below the existing methods.