# An Internal Learning Approach to Video Inpainting

arXiv: https://arxiv.org/abs/1909.07957 · HTML: https://mlsanity.com/paper/1909.07957/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Sep 2019.

## Built on by (most cited first, 30 of 33)

- [ProPainter: Improving Propagation and Transformer for Video Inpainting](https://mlsanity.com/paper/2309.03897/) (2023)
- [HNeRV: A Hybrid Neural Representation for Videos](https://mlsanity.com/paper/2304.02633/) (2023)
- [FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting](https://mlsanity.com/paper/2109.02974/) (2021)
- [Flow-Guided Transformer for Video Inpainting](https://mlsanity.com/paper/2208.06768/) (2022)
- [Drop the GAN: In Defense of Patches Nearest Neighbors as Single Image Generative Models](https://mlsanity.com/paper/2103.15545/) (2021)
- [Deep Learning-based Image and Video Inpainting: A Survey](https://mlsanity.com/paper/2401.03395/) (2024)
- [SEN12MS-CR-TS: A Remote Sensing Data Set for Multi-modal Multi-temporal Cloud Removal](https://mlsanity.com/paper/2201.09613/) (2022)
- [Occlusion-Aware Video Object Inpainting](https://mlsanity.com/paper/2108.06765/) (2021)
- [Decoupled Spatial-Temporal Transformer for Video Inpainting](https://mlsanity.com/paper/2104.06637/) (2021)
- [Video Demoireing with Relation-Based Temporal Consistency](https://mlsanity.com/paper/2204.02957/) (2022)
- [Internal Video Inpainting by Implicit Long-range Propagation](https://mlsanity.com/paper/2108.01912/) (2021)
- [Deep Video Prior for Video Consistency and Propagation](https://mlsanity.com/paper/2201.11632/) (2022)
- [SinIR: Efficient General Image Manipulation with Single Image Reconstruction](https://mlsanity.com/paper/2106.07140/) (2021)
- [Diverse Generation from a Single Video Made Possible](https://mlsanity.com/paper/2109.08591/) (2021)
- [The DEVIL is in the Details: A Diagnostic Evaluation Benchmark for Video Inpainting](https://mlsanity.com/paper/2105.05332/) (2021)
- [Flow-Guided Video Inpainting with Scene Templates](https://mlsanity.com/paper/2108.12845/) (2021)
- [Deep Video Inpainting Detection](https://mlsanity.com/paper/2101.11080/) (2021)
- [Co-attention Propagation Network for Zero-Shot Video Object Segmentation](https://mlsanity.com/paper/2304.03910/) (2023)
- [Semi-Supervised Video Inpainting with Cycle Consistency Constraints](https://mlsanity.com/paper/2208.06807/) (2022)
- [U-TILISE: A Sequence-to-sequence Model for Cloud Removal in Optical Satellite Time Series](https://mlsanity.com/paper/2305.13277/) (2023)
- [A Review on Visual Privacy Preservation Techniques for Active and Assisted Living](https://mlsanity.com/paper/2112.09422/) (2021)
- [Exploiting Optical Flow Guidance for Transformer-Based Video Inpainting](https://mlsanity.com/paper/2301.10048/) (2023)
- [Object-WIPER : Training-Free Object and Associated Effect Removal in Videos](https://mlsanity.com/paper/2601.06391/) (2026)
- [Video Dynamics Prior: An Internal Learning Approach for Robust Video Enhancements](https://mlsanity.com/paper/2312.07835/) (2023)
- [BVINet: Unlocking Blind Video Inpainting with Zero Annotations](https://mlsanity.com/paper/2502.01181/) (2025)
- [PIP: Positional-encoding Image Prior](https://mlsanity.com/paper/2211.14298/) (2022)
- [From Understanding to Erasing: Towards Complete and Stable Video Object Removal](https://mlsanity.com/paper/2604.01693/) (2026)
- [Spatial-Temporal Residual Aggregation for High Resolution Video Inpainting](https://mlsanity.com/paper/2111.03574/) (2021)
- [Infusion: internal diffusion for inpainting of dynamic textures and complex motion](https://mlsanity.com/paper/2311.01090/) (2023)
- [Untrained Neural Nets for Snapshot Compressive Imaging: Theory and Algorithms](https://mlsanity.com/paper/2406.03694/) (2024)

## Abstract (arXiv)

We propose a novel video inpainting algorithm that simultaneously hallucinates missing appearance and motion (optical flow) information, building upon the recent 'Deep Image Prior' (DIP) that exploits convolutional network architectures to enforce plausible texture in static images. In extending DIP to video we make two important contributions. First, we show that coherent video inpainting is possible without a priori training. We take a generative approach to inpainting based on internal (within-video) learning without reliance upon an external corpus of visual data to train a one-size-fits-all model for the large space of general videos. Second, we show that such a framework can jointly generate both appearance and flow, whilst exploiting these complementary modalities to ensure mutual consistency. We show that leveraging appearance statistics specific to each video achieves visually plausible results whilst handling the challenging problem of long-term consistency.