# Hierarchical Dynamic Image Harmonization arXiv: https://arxiv.org/abs/2211.08639 ยท HTML: https://mlsanity.com/paper/2211.08639/ ## Key facts - Cited by 12 papers in the map (data through 14 Oct 2026). - Builds on 28 of its 45 references that are in the map. - First posted on arXiv 16 Nov 2022. ## Builds on (most cited first, 28 of 28) - [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019) - [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018) - [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017) - [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017) - [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018) - [ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1910.03151/) (2019) - [Dynamic Neural Networks: A Survey](https://mlsanity.com/paper/2102.04906/) (2021) - [Universal Style Transfer via Feature Transforms](https://mlsanity.com/paper/1705.08086/) (2017) - [Arbitrary Style Transfer with Style-Attentional Networks](https://mlsanity.com/paper/1812.02342/) (2018) - [EditGAN: High-Precision Semantic Image Editing](https://mlsanity.com/paper/2111.03186/) (2021) - [DoveNet: Deep Image Harmonization via Domain Verification](https://mlsanity.com/paper/1911.13239/) (2019) - [Deep Image Harmonization](https://mlsanity.com/paper/1703.00069/) (2017) - [Context-aware Feature Generation for Zero-shot Semantic Segmentation](https://mlsanity.com/paper/2008.06893/) (2020) - [Dynamic Instance Normalization for Arbitrary Style Transfer](https://mlsanity.com/paper/1911.06953/) (2019) - [Batch-Instance Normalization for Adaptively Style-Invariant Neural Networks](https://mlsanity.com/paper/1805.07925/) (2018) - [Improving the Harmony of the Composite Image by Spatial-Separated Attention Module](https://mlsanity.com/paper/1907.06406/) (2019) - [Region-aware Adaptive Instance Normalization for Image Harmonization](https://mlsanity.com/paper/2106.02853/) (2021) - [Harmonizer: Learning to Perform White-Box Image and Video Harmonization](https://mlsanity.com/paper/2207.01322/) (2022) - [Semantic Scene Completion via Integrating Instances and Scene in-the-Loop](https://mlsanity.com/paper/2104.03640/) (2021) - [High-Resolution Image Harmonization via Collaborative Dual Transformations](https://mlsanity.com/paper/2109.06671/) (2021) - [Foreground-aware Semantic Representations for Image Harmonization](https://mlsanity.com/paper/2006.00809/) (2020) - [BargainNet: Background-Guided Domain Translation for Image Harmonization](https://mlsanity.com/paper/2009.09169/) (2020) - [Dynamic Region-Aware Convolution](https://mlsanity.com/paper/2003.12243/) (2020) - [DCCF: Deep Comprehensible Color Filter Learning Framework for High-Resolution Image Harmonization](https://mlsanity.com/paper/2207.04788/) (2022) - [Spatial-Separated Curve Rendering Network for Efficient and High-Resolution Image Harmonization](https://mlsanity.com/paper/2109.05750/) (2021) - [SCS-Co: Self-Consistent Style Contrastive Learning for Image Harmonization](https://mlsanity.com/paper/2204.13962/) (2022) - [Not All Voxels Are Equal: Semantic Scene Completion from the Point-Voxel Perspective](https://mlsanity.com/paper/2112.12925/) (2021) - [A2: Efficient Automated Attacker for Boosting Adversarial Training](https://mlsanity.com/paper/2210.03543/) (2022) ## Built on by (most cited first, 12 of 12) - [AnyDoor: Zero-shot Object-level Image Customization](https://mlsanity.com/paper/2307.09481/) (2023) - [Zero-shot Image Editing with Reference Imitation](https://mlsanity.com/paper/2406.07547/) (2024) - [DiffUTE: Universal Text Editing Diffusion Model](https://mlsanity.com/paper/2305.10825/) (2023) - [CustomNet: Zero-shot Object Customization with Variable-Viewpoints in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2310.19784/) (2023) - [Training-Free Semantic Video Composition via Pre-trained Diffusion Model](https://mlsanity.com/paper/2401.09195/) (2024) - [DiffHarmony: Latent Diffusion Model Meets Image Harmonization](https://mlsanity.com/paper/2404.06139/) (2024) - [AnyLogo: Symbiotic Subject-Driven Diffusion System with Gemini Status](https://mlsanity.com/paper/2409.17740/) (2024) - [Segment Anything Model Meets Image Harmonization](https://mlsanity.com/paper/2312.12729/) (2023) - [PHNet: Patch-based Normalization for Portrait Harmonization](https://mlsanity.com/paper/2402.17561/) (2024) - [Harmonizing Attention: Training-free Texture-aware Geometry Transfer](https://mlsanity.com/paper/2408.10846/) (2024) - [Retrieval Augmented Image Harmonization](https://mlsanity.com/paper/2412.13916/) (2024) - [VSDiffusion: Taming Ill-Posed Shadow Generation via Visibility-Constrained Diffusion](https://mlsanity.com/paper/2603.08020/) (2026) ## Abstract (arXiv) Image harmonization is a critical task in computer vision, which aims to adjust the foreground to make it compatible with the background. Recent works mainly focus on using global transformations (i.e., normalization and color curve rendering) to achieve visual consistency. However, these models ignore local visual consistency and their huge model sizes limit their harmonization ability on edge devices. In this paper, we propose a hierarchical dynamic network (HDNet) to adapt features from local to global view for better feature transformation in efficient image harmonization. Inspired by the success of various dynamic models, local dynamic (LD) module and mask-aware global dynamic (MGD) module are proposed in this paper. Specifically, LD matches local representations between the foreground and background regions based on semantic similarities, then adaptively adjust every foreground local representation according to the appearance of its K-nearest neighbor background regions. In this way, LD can produce more realistic images at a more fine-grained level, and simultaneously enjoy the characteristic of semantic alignment. The MGD effectively applies distinct convolution to the foreground and background, learning the representations of foreground and background regions as well as their correlations to the global harmonization, facilitating local visual consistency for the images much more efficiently. Experimental results demonstrate that the proposed HDNet significantly reduces the total model parameters by more than 80% compared to previous methods, while still attaining state-of-the-art performance on the popular iHarmony4 dataset. Notably, the HDNet achieves a 4% improvement in PSNR and a 19% reduction in MSE compared to the prior state-of-the-art methods.