# Temporally Coherent Video Harmonization Using Adversarial Networks

arXiv: https://arxiv.org/abs/1809.01372 · HTML: https://mlsanity.com/paper/1809.01372/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Sep 2018.

## Built on by (most cited first, 10 of 10)

- [Making Images Real Again: A Comprehensive Survey on Deep Image Composition](https://mlsanity.com/paper/2106.14490/) (2021)
- [Harmonizer: Learning to Perform White-Box Image and Video Harmonization](https://mlsanity.com/paper/2207.01322/) (2022)
- [DeCo: Decoupled Human-Centered Diffusion Video Editing with Motion Consistency](https://mlsanity.com/paper/2408.07481/) (2024)
- [DiffusionHarmonizer: Bridging Neural Reconstruction and Photorealistic Simulation with Online Diffusion Enhancer](https://mlsanity.com/paper/2602.24096/) (2026)
- [Deep Video Harmonization with Color Mapping Consistency](https://mlsanity.com/paper/2205.00687/) (2022)
- [Anything in Any Scene: Photorealistic Video Object Insertion](https://mlsanity.com/paper/2401.17509/) (2024)
- [Training-Free Semantic Video Composition via Pre-trained Diffusion Model](https://mlsanity.com/paper/2401.09195/) (2024)
- [HYouTube: Video Harmonization Dataset](https://mlsanity.com/paper/2109.08809/) (2021)
- [Painterly Image Harmonization via Adversarial Residual Learning](https://mlsanity.com/paper/2311.08646/) (2023)
- [Interpretable Deepfake Detection in Videos via Explicit Forensic Features and Temporal Modeling](https://mlsanity.com/paper/2610.03380/) (2026)

## Abstract (arXiv)

Compositing is one of the most important editing operations for images and videos. The process of improving the realism of composite results is often called harmonization. Previous approaches for harmonization mainly focus on images. In this work, we take one step further to attack the problem of video harmonization. Specifically, we train a convolutional neural network in an adversarial way, exploiting a pixel-wise disharmony discriminator to achieve more realistic harmonized results and introducing a temporal loss to increase temporal consistency between consecutive harmonized frames. Thanks to the pixel-wise disharmony discriminator, we are also able to relieve the need of input foreground masks. Since existing video datasets which have ground-truth foreground masks and optical flows are not sufficiently large, we propose a simple yet efficient method to build up a synthetic dataset supporting supervised training of the proposed adversarial network. Experiments show that training on our synthetic dataset generalizes well to the real-world composite dataset. Also, our method successfully incorporates temporal consistency during training and achieves more harmonious results than previous methods.