# Video Inpainting Localization with Contrastive Learning

arXiv: https://arxiv.org/abs/2406.17628 · HTML: https://mlsanity.com/paper/2406.17628/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 36 references that are in the map.
- First posted on arXiv 25 Jun 2024.

## Builds on (most cited first, 19 of 19)

- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild](https://mlsanity.com/paper/1810.11981/) (2018)
- [MOSE: A New Dataset for Video Object Segmentation in Complex Scenes](https://mlsanity.com/paper/2302.01872/) (2023)
- [YouTube-VOS: Sequence-to-Sequence Video Object Segmentation](https://mlsanity.com/paper/1809.00461/) (2018)
- [UniFormer: Unifying Convolution and Self-attention for Visual Recognition](https://mlsanity.com/paper/2201.09450/) (2022)
- [Learning Joint Spatial-Temporal Transformations for Video Inpainting](https://mlsanity.com/paper/2007.10247/) (2020)
- [See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networks](https://mlsanity.com/paper/2001.06810/) (2020)
- [Towards An End-to-End Framework for Flow-Guided Video Inpainting](https://mlsanity.com/paper/2204.02663/) (2022)
- [What makes instance discrimination good for transfer learning?](https://mlsanity.com/paper/2006.06606/) (2020)
- [Deep Video Inpainting](https://mlsanity.com/paper/1905.01639/) (2019)
- [FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting](https://mlsanity.com/paper/2109.02974/) (2021)
- [Copy-and-Paste Networks for Deep Video Inpainting](https://mlsanity.com/paper/1908.11587/) (2019)
- [Generate, Segment and Refine: Towards Generic Manipulation Segmentation](https://mlsanity.com/paper/1811.09729/) (2018)
- [Pre-training-free Image Manipulation Localization through Non-Mutually Exclusive Contrastive Learning](https://mlsanity.com/paper/2309.14900/) (2023)
- [Mask-Free Video Instance Segmentation](https://mlsanity.com/paper/2303.15904/) (2023)
- [CFL-Net: Image Forgery Localization Using Contrastive Learning](https://mlsanity.com/paper/2210.02182/) (2022)
- [Deep Video Inpainting Detection](https://mlsanity.com/paper/2101.11080/) (2021)
- [VideoFACT: Detecting Video Forgeries Using Attention, Scene Context, and Forensic Traces](https://mlsanity.com/paper/2211.15775/) (2022)

## Built on by (most cited first, 1 of 1)

- [RelayFormer: A Unified Local-Global Attention Framework for Scalable Image and Video Manipulation Localization](https://mlsanity.com/paper/2508.09459/) (2025)

## Abstract (arXiv)

Deep video inpainting is typically used as malicious manipulation to remove important objects for creating fake videos. It is significant to identify the inpainted regions blindly. This letter proposes a simple yet effective forensic scheme for Video Inpainting LOcalization with ContrAstive Learning (ViLocal). Specifically, a 3D Uniformer encoder is applied to the video noise residual for learning effective spatiotemporal forensic features. To enhance the discriminative power, supervised contrastive learning is adopted to capture the local inconsistency of inpainted videos through attracting/repelling the positive/negative pristine and forged pixel pairs. A pixel-wise inpainting localization map is yielded by a lightweight convolution decoder with a specialized two-stage training strategy. To prepare enough training samples, we build a video object segmentation dataset of 2500 videos with pixel-level annotations per frame. Extensive experimental results validate the superiority of ViLocal over state-of-the-arts. Code and dataset will be available at https://github.com/multimediaFor/ViLocal.