# Free-form Video Inpainting with 3D Gated Convolution and Temporal PatchGAN

arXiv: https://arxiv.org/abs/1904.10247 · HTML: https://mlsanity.com/paper/1904.10247/

## Key facts

- Cited by 87 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Apr 2019.

## Built on by (most cited first, 30 of 87)

- [ProPainter: Improving Propagation and Transformer for Video Inpainting](https://mlsanity.com/paper/2309.03897/) (2023)
- [Towards An End-to-End Framework for Flow-Guided Video Inpainting](https://mlsanity.com/paper/2204.02663/) (2022)
- [FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting](https://mlsanity.com/paper/2109.02974/) (2021)
- [VideoPainter: Any-length Video Inpainting and Editing with Plug-and-Play Context Control](https://mlsanity.com/paper/2503.05639/) (2025)
- [Flow-Guided Transformer for Video Inpainting](https://mlsanity.com/paper/2208.06768/) (2022)
- [Deep Learning-based Image and Video Inpainting: A Survey](https://mlsanity.com/paper/2401.03395/) (2024)
- [Generative Omnimatte: Learning to Decompose Video into Layers](https://mlsanity.com/paper/2411.16683/) (2024)
- [ROSE: Remove Objects with Side Effects in Videos](https://mlsanity.com/paper/2508.18633/) (2025)
- [Progressive Temporal Feature Alignment Network for Video Inpainting](https://mlsanity.com/paper/2104.03507/) (2021)
- [Towards Language-Driven Video Inpainting via Multimodal Large Language Models](https://mlsanity.com/paper/2401.10226/) (2024)
- [Dense Deep Unfolding Network with 3D-CNN Prior for Snapshot Compressive Imaging](https://mlsanity.com/paper/2109.06548/) (2021)
- [Bringing Old Films Back to Life](https://mlsanity.com/paper/2203.17276/) (2022)
- [Occlusion-Aware Video Object Inpainting](https://mlsanity.com/paper/2108.06765/) (2021)
- [Decoupled Spatial-Temporal Transformer for Video Inpainting](https://mlsanity.com/paper/2104.06637/) (2021)
- [ClothFormer:Taming Video Virtual Try-on in All Module](https://mlsanity.com/paper/2204.12151/) (2022)
- [Video Demoireing with Relation-Based Temporal Consistency](https://mlsanity.com/paper/2204.02957/) (2022)
- [High-resolution Depth Maps Imaging via Attention-based Hierarchical Multi-modal Fusion](https://mlsanity.com/paper/2104.01530/) (2021)
- [Error Compensation Framework for Flow-Guided Video Inpainting](https://mlsanity.com/paper/2207.10391/) (2022)
- [Internal Video Inpainting by Implicit Long-range Propagation](https://mlsanity.com/paper/2108.01912/) (2021)
- [EfficientSCI: Densely Connected Network with Space-time Factorization for Large-scale Video Snapshot Compressive Imaging](https://mlsanity.com/paper/2305.10006/) (2023)
- [Free-form tumor synthesis in computed tomography images via richer generative adversarial network](https://mlsanity.com/paper/2104.09701/) (2021)
- [OmnimatteZero: Fast Training-free Omnimatte with Pre-trained Video Diffusion Models](https://mlsanity.com/paper/2503.18033/) (2025)
- [NTIRE 2026 Challenge on Bitstream-Corrupted Video Restoration: Methods and Results](https://mlsanity.com/paper/2604.06945/) (2026)
- [Bitstream-Corrupted Video Recovery: A Novel Benchmark Dataset and Method](https://mlsanity.com/paper/2309.13890/) (2023)
- [EffectErase: Joint Video Object Removal and Insertion for High-Quality Effect Erasing](https://mlsanity.com/paper/2603.19224/) (2026)
- [The DEVIL is in the Details: A Diagnostic Evaluation Benchmark for Video Inpainting](https://mlsanity.com/paper/2105.05332/) (2021)
- [Flow-Guided Video Inpainting with Scene Templates](https://mlsanity.com/paper/2108.12845/) (2021)
- [Deep Video Inpainting Detection](https://mlsanity.com/paper/2101.11080/) (2021)
- [PISCO: Precise Video Instance Insertion with Sparse Control](https://mlsanity.com/paper/2602.08277/) (2026)
- [Let's See Clearly: Contaminant Artifact Removal for Moving Cameras](https://mlsanity.com/paper/2104.08852/) (2021)

## Abstract (arXiv)

Free-form video inpainting is a very challenging task that could be widely used for video editing such as text removal. Existing patch-based methods could not handle non-repetitive structures such as faces, while directly applying image-based inpainting models to videos will result in temporal inconsistency (see http://bit.ly/2Fu1n6b ). In this paper, we introduce a deep learn-ing based free-form video inpainting model, with proposed 3D gated convolutions to tackle the uncertainty of free-form masks and a novel Temporal PatchGAN loss to enhance temporal consistency. In addition, we collect videos and design a free-form mask generation algorithm to build the free-form video inpainting (FVI) dataset for training and evaluation of video inpainting models. We demonstrate the benefits of these components and experiments on both the FaceForensics and our FVI dataset suggest that our method is superior to existing ones. Related source code, full-resolution result videos and the FVI dataset could be found on Github https://github.com/amjltc295/Free-Form-Video-Inpainting .