Foreground-aware Semantic Representations for Image Harmonization
Not on the map. Why?
Abstract (arXiv)
Image harmonization is an important step in photo editing to achieve visual consistency in composite images by adjusting the appearances of foreground to make it compatible with background. Previous approaches to harmonize composites are based on training of encoder-decoder networks from scratch, which makes it challenging for a neural network to learn a high-level representation of objects. We propose a novel architecture to utilize the space of high-level features learned by a pre-trained classification network. We create our models as a combination of existing encoder-decoder architectures and a pre-trained foreground-aware deep high-resolution network. We extensively evaluate the proposed method on existing image harmonization benchmark and set up a new state-of-the-art in terms of MSE and PSNR metrics. The code and trained models are available at urlhttps://github.com/saic-vul/imageharmonization.
Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Jun 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 8; 2022: 17; 2023: 14; 2024: 9; 2025: 3; 2026: 2.
53 papers.
Sort:
By area:
- 2021Making Images Real Again: A Comprehensive Survey on Deep Image Compositioncited by 63
- 2022Harmonizer: Learning to Perform White-Box Image and Video Harmonizationcited by 61
- 2021High-Resolution Image Harmonization via Collaborative Dual Transformationscited by 55
- 2022DCCF: Deep Comprehensible Color Filter Learning Framework for High-Resolution Image Harmonizationcited by 46
- 2023ControlCom: Controllable Image Composition using Diffusion Modelcited by 43
- 2021Spatial-Separated Curve Rendering Network for Efficient and High-Resolution Image Harmonizationcited by 25
- 2022SCS-Co: Self-Consistent Style Contrastive Learning for Image Harmonizationcited by 24
- 2023Paste, Inpaint and Harmonize via Denoising: Subject-Driven Image Editing with Pre-Trained Diffusion Modelcited by 20
- 2023Dense Pixel-to-Pixel Harmonization via Continuous Image Representationcited by 19
- 2021Learning Interactive Driving Policies via Data-driven Simulationcited by 17
- 2023DreamCom: Finetuning Text-guided Inpainting Model for Image Compositioncited by 17
- 2024FreeCompose: Generic Zero-Shot Image Composition with Diffusion Priorcited by 16
38 more
- 2023PixHt-Lab: Pixel Height Based Light Effect Generation for Image Compositingcited by 15
- 2024DeCo: Decoupled Human-Centered Diffusion Video Editing with Motion Consistencycited by 13
- 2022Fast Object Placement Assessmentcited by 12
- 2022Hierarchical Dynamic Image Harmonizationcited by 12
- 2023LEMaRT: Label-Efficient Masked Region Transform for Image Harmonizationcited by 12
- 2023Deep Image Harmonization in Dual Color Spacescited by 12
- 2021Deep Image Harmonization by Bridging the Reality Gapcited by 11
- 2022Painterly Image Harmonization in Dual Domainscited by 11
- 2022Deep Video Harmonization with Color Mapping Consistencycited by 9
- 2022FRIH: Fine-grained Region-aware Image Harmonizationcited by 9
- 2023Zero-Shot Image Harmonization with Generative Model Priorcited by 8
- 2025MV-CoLight: Efficient Object Compositing with Consistent Lighting and Shadow Generationcited by 8
- 2022Image Harmonization by Matching Regional Referencescited by 6
- 2023Deep Image Harmonization with Learnable Augmentationcited by 5
- 2023Learning Global-aware Kernel for Image Harmonizationcited by 4
- 2023Deep Image Harmonization with Globally Guided Feature Transformation and Relation Distillationcited by 4
- 2024Privacy-Preserving Synthetic Continual Semantic Segmentation for Robotic Surgerycited by 4
- 2025MagicQuillV2: Precise and Interactive Image Editing with Layered Visual Cuescited by 4
- 2023Painterly Image Harmonization by Learning from Painterly Objectscited by 3
- 2021Deep Portrait Lighting Enhancement with 3D Guidancecited by 2
- 2021SIDNet: Learning Shading-aware Illumination Descriptor for Image Harmonizationcited by 2
- 2022Interactive Portrait Harmonizationcited by 2
- 2022Dual-Domain Image Synthesis using Segmentation-Guided GANcited by 2
- 2022Inharmonious Region Localization by Magnifying Domain Discrepancycited by 2
- 2024GroupDiff: Diffusion-based Group Portrait Editingcited by 2
- 2022Image Harmonization with Region-wise Contrastive Learningcited by 1
- 2022Inharmonious Region Localization with Auxiliary Style Featurecited by 1
- 2023Progressive Painterly Image Harmonization from Low-level Styles to High-level Stylescited by 1
- 2024Diverse Image Harmonizationcited by 1
- 2021HYouTube: Video Harmonization Datasetcited by 0
- 2022CcHarmony: Color-checker based Image Harmonization Datasetcited by 0
- 2022Inharmonious Region Localization via Recurrent Self-Reasoningcited by 0
- 2022AMICO: Amodal Instance Compositioncited by 0
- 2023Painterly Image Harmonization via Adversarial Residual Learningcited by 0
- 2024PHNet: Patch-based Normalization for Portrait Harmonizationcited by 0
- 2024DepGAN: Leveraging Depth Maps for Handling Occlusions and Transparency in Image Compositioncited by 0
- 2024Harmonizing Attention: Training-free Texture-aware Geometry Transfercited by 0
- 2024Retrieval Augmented Image Harmonizationcited by 0
50 most cited of 53 All 53 (JSON)
- 2026Hybrid Generative-Discriminative Object Placement24 Aug 2026
- 2026A Non-Reference Diffusion-Based Restoration Framework for Landsat 7 ETM+ SLC-off Imagery in Antarctica20 May 2026
- 2025MagicQuillV2: Precise and Interactive Image Editing with Layered Visual Cues2 Dec 2025
- 2025Region-to-Region: Enhancing Generative Image Harmonization with Adaptive Regional Injection13 Aug 2025
- 2025MV-CoLight: Efficient Object Compositing with Consistent Lighting and Shadow Generation27 May 2025
- 2024Retrieval Augmented Image Harmonization18 Dec 2024
- 2024GroupDiff: Diffusion-based Group Portrait Editing22 Sep 2024
- 2024Harmonizing Attention: Training-free Texture-aware Geometry Transfer19 Aug 2024
- 2024DeCo: Decoupled Human-Centered Diffusion Video Editing with Motion Consistency14 Aug 2024
- 2024Diverse Image Harmonization22 Jul 2024
- 2024DepGAN: Leveraging Depth Maps for Handling Occlusions and Transparency in Image Composition16 Jul 2024
- 2024FreeCompose: Generic Zero-Shot Image Composition with Diffusion Prior6 Jul 2024
38 more
- 2024PHNet: Patch-based Normalization for Portrait Harmonization27 Feb 2024
- 2024Privacy-Preserving Synthetic Continual Semantic Segmentation for Robotic Surgery8 Feb 2024
- 2023Progressive Painterly Image Harmonization from Low-level Styles to High-level Styles15 Dec 2023
- 2023Painterly Image Harmonization by Learning from Painterly Objects15 Dec 2023
- 2023Painterly Image Harmonization via Adversarial Residual Learning15 Nov 2023
- 2023DreamCom: Finetuning Text-guided Inpainting Model for Image Composition27 Sep 2023
- 2023ControlCom: Controllable Image Composition using Diffusion Model19 Aug 2023
- 2023Deep Image Harmonization in Dual Color Spaces5 Aug 2023
- 2023Deep Image Harmonization with Learnable Augmentation1 Aug 2023
- 2023Deep Image Harmonization with Globally Guided Feature Transformation and Relation Distillation1 Aug 2023
- 2023Zero-Shot Image Harmonization with Generative Model Prior17 Jul 2023
- 2023Paste, Inpaint and Harmonize via Denoising: Subject-Driven Image Editing with Pre-Trained Diffusion Model13 Jun 2023
- 2023Learning Global-aware Kernel for Image Harmonization19 May 2023
- 2023LEMaRT: Label-Efficient Masked Region Transform for Image Harmonization25 Apr 2023
- 2023Dense Pixel-to-Pixel Harmonization via Continuous Image Representation3 Mar 2023
- 2023PixHt-Lab: Pixel Height Based Light Effect Generation for Image Compositing28 Feb 2023
- 2022Painterly Image Harmonization in Dual Domains17 Dec 2022
- 2022Hierarchical Dynamic Image Harmonization16 Nov 2022
- 2022AMICO: Amodal Instance Composition11 Oct 2022
- 2022Inharmonious Region Localization via Recurrent Self-Reasoning5 Oct 2022
- 2022Inharmonious Region Localization with Auxiliary Style Feature5 Oct 2022
- 2022Inharmonious Region Localization by Magnifying Domain Discrepancy30 Sep 2022
- 2022DCCF: Deep Comprehensible Color Filter Learning Framework for High-Resolution Image Harmonization11 Jul 2022
- 2022Harmonizer: Learning to Perform White-Box Image and Video Harmonization4 Jul 2022
- 2022CcHarmony: Color-checker based Image Harmonization Dataset1 Jun 2022
- 2022Fast Object Placement Assessment28 May 2022
- 2022Image Harmonization with Region-wise Contrastive Learning27 May 2022
- 2022FRIH: Fine-grained Region-aware Image Harmonization13 May 2022
- 2022Deep Video Harmonization with Color Mapping Consistency2 May 2022
- 2022SCS-Co: Self-Consistent Style Contrastive Learning for Image Harmonization29 Apr 2022
- 2022Dual-Domain Image Synthesis using Segmentation-Guided GAN19 Apr 2022
- 2022Image Harmonization by Matching Regional References10 Apr 2022
- 2022Interactive Portrait Harmonization15 Mar 2022
- 2021SIDNet: Learning Shading-aware Illumination Descriptor for Image Harmonization2 Dec 2021
- 2021Learning Interactive Driving Policies via Data-driven Simulation23 Nov 2021
- 2021HYouTube: Video Harmonization Dataset18 Sep 2021
- 2021High-Resolution Image Harmonization via Collaborative Dual Transformations14 Sep 2021
- 2021Spatial-Separated Curve Rendering Network for Efficient and High-Resolution Image Harmonization13 Sep 2021
50 newest of 53 All 53 (JSON)
- 2023ControlCom: Controllable Image Composition using Diffusion Modelcited by 43
- 2023Paste, Inpaint and Harmonize via Denoising: Subject-Driven Image Editing with Pre-Trained Diffusion Modelcited by 20
- 2024FreeCompose: Generic Zero-Shot Image Composition with Diffusion Priorcited by 16
- 2024DeCo: Decoupled Human-Centered Diffusion Video Editing with Motion Consistencycited by 13
- 2025MagicQuillV2: Precise and Interactive Image Editing with Layered Visual Cuescited by 4
- 2025MagicQuillV2: Precise and Interactive Image Editing with Layered Visual Cues2 Dec 2025
- 2024DeCo: Decoupled Human-Centered Diffusion Video Editing with Motion Consistency14 Aug 2024
- 2024FreeCompose: Generic Zero-Shot Image Composition with Diffusion Prior6 Jul 2024
- 2023ControlCom: Controllable Image Composition using Diffusion Model19 Aug 2023
- 2023Paste, Inpaint and Harmonize via Denoising: Subject-Driven Image Editing with Pre-Trained Diffusion Model13 Jun 2023
- 2021Making Images Real Again: A Comprehensive Survey on Deep Image Compositioncited by 63
- 2022Harmonizer: Learning to Perform White-Box Image and Video Harmonizationcited by 61
- 2021High-Resolution Image Harmonization via Collaborative Dual Transformationscited by 55
- 2022DCCF: Deep Comprehensible Color Filter Learning Framework for High-Resolution Image Harmonizationcited by 46
- 2021Spatial-Separated Curve Rendering Network for Efficient and High-Resolution Image Harmonizationcited by 25
- 2022SCS-Co: Self-Consistent Style Contrastive Learning for Image Harmonizationcited by 24
- 2023Dense Pixel-to-Pixel Harmonization via Continuous Image Representationcited by 19
- 2023DreamCom: Finetuning Text-guided Inpainting Model for Image Compositioncited by 17
- 2023PixHt-Lab: Pixel Height Based Light Effect Generation for Image Compositingcited by 15
- 2022Fast Object Placement Assessmentcited by 12
- 2022Hierarchical Dynamic Image Harmonizationcited by 12
- 2023LEMaRT: Label-Efficient Masked Region Transform for Image Harmonizationcited by 12
12 most cited of 47 All 47 (JSON)
- 2026Hybrid Generative-Discriminative Object Placement24 Aug 2026
- 2026A Non-Reference Diffusion-Based Restoration Framework for Landsat 7 ETM+ SLC-off Imagery in Antarctica20 May 2026
- 2025Region-to-Region: Enhancing Generative Image Harmonization with Adaptive Regional Injection13 Aug 2025
- 2025MV-CoLight: Efficient Object Compositing with Consistent Lighting and Shadow Generation27 May 2025
- 2024Retrieval Augmented Image Harmonization18 Dec 2024
- 2024GroupDiff: Diffusion-based Group Portrait Editing22 Sep 2024
- 2024Harmonizing Attention: Training-free Texture-aware Geometry Transfer19 Aug 2024
- 2024Diverse Image Harmonization22 Jul 2024
- 2024DepGAN: Leveraging Depth Maps for Handling Occlusions and Transparency in Image Composition16 Jul 2024
- 2024PHNet: Patch-based Normalization for Portrait Harmonization27 Feb 2024
- 2024Privacy-Preserving Synthetic Continual Semantic Segmentation for Robotic Surgery8 Feb 2024
- 2023Progressive Painterly Image Harmonization from Low-level Styles to High-level Styles15 Dec 2023
12 newest of 47 All 47 (JSON)