# VPN: Video Provenance Network for Robust Content Attribution

arXiv: https://arxiv.org/abs/2109.10038 · HTML: https://mlsanity.com/paper/2109.10038/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 67 references that are in the map.
- First posted on arXiv 21 Sep 2021.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [CNN-generated images are surprisingly easy to spot... for now](https://mlsanity.com/paper/1912.11035/) (2019)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [VGGSound: A Large-scale Audio-Visual Dataset](https://mlsanity.com/paper/2004.14368/) (2020)
- [The Deepfake Detection Challenge (DFDC) Preview Dataset](https://mlsanity.com/paper/1910.08854/) (2019)
- [Artificial Fingerprinting for Generative Models: Rooting Deepfake Attribution in Training Data](https://mlsanity.com/paper/2007.08457/) (2020)
- [Deep Image Retrieval: Learning global representations for image search](https://mlsanity.com/paper/1604.01325/) (2016)
- [HashNet: Deep Learning to Hash by Continuation](https://mlsanity.com/paper/1702.00758/) (2017)
- [Central Similarity Quantization for Efficient Image and Video Retrieval](https://mlsanity.com/paper/1908.00347/) (2019)
- [Feature Learning based Deep Supervised Hashing with Pairwise Labels](https://mlsanity.com/paper/1511.03855/) (2015)
- [Detecting Photoshopped Faces by Scripting Photoshop](https://mlsanity.com/paper/1906.05856/) (2019)
- [Convolutional Recurrent Neural Networks for Music Classification](https://mlsanity.com/paper/1609.04243/) (2016)
- [The 2021 Image Similarity Dataset and Challenge](https://mlsanity.com/paper/2106.09672/) (2021)
- [Deep Supervised Discrete Hashing](https://mlsanity.com/paper/1705.10999/) (2017)
- [Asymmetric Deep Supervised Hashing](https://mlsanity.com/paper/1707.08325/) (2017)
- [Hashing for Similarity Search: A Survey](https://mlsanity.com/paper/1408.2927/) (2014)
- [Sample-level Deep Convolutional Neural Networks for Music Auto-tagging Using Raw Waveforms](https://mlsanity.com/paper/1703.01789/) (2017)
- [Image Provenance Analysis at Scale](https://mlsanity.com/paper/1801.06510/) (2018)
- [OSCAR-Net: Object-centric Scene Graph Attention for Image Attribution](https://mlsanity.com/paper/2108.03541/) (2021)
- [Deep convolutional neural networks for predominant instrument recognition in polyphonic music](https://mlsanity.com/paper/1605.09507/) (2016)
- [A Tutorial on Deep Learning for Music Information Retrieval](https://mlsanity.com/paper/1709.04396/) (2017)
- [ARCHANGEL: Trusted Archives of Digital Public Documents](https://mlsanity.com/paper/1804.08342/) (2018)
- [Push for Quantization: Deep Fisher Hashing](https://mlsanity.com/paper/1909.00206/) (2019)
- [Revisiting the problem of audio-based hit song prediction using convolutional neural networks](https://mlsanity.com/paper/1704.01280/) (2017)
- [An End-to-End Neural Network for Polyphonic Piano Music Transcription](https://mlsanity.com/paper/1508.01774/) (2015)

## Built on by (most cited first, 7 of 7)

- [RoSteALS: Robust Steganography using Autoencoder Latent Space](https://mlsanity.com/paper/2304.03400/) (2023)
- [RepMix: Representation Mixing for Robust Attribution of Synthesized Images](https://mlsanity.com/paper/2207.02063/) (2022)
- [InvisMark: Invisible and Robust Watermarking for AI-generated Image Provenance](https://mlsanity.com/paper/2411.07795/) (2024)
- [Audio-Visual Contrastive Learning with Temporal Self-Supervision](https://mlsanity.com/paper/2302.07702/) (2023)
- [VADER: Video Alignment Differencing and Retrieval](https://mlsanity.com/paper/2303.13193/) (2023)
- [ARIA: Adversarially Robust Image Attribution for Content Provenance](https://mlsanity.com/paper/2202.12860/) (2022)
- [SAiW: Source-Attributable Invisible Watermarking for Proactive Deepfake Defense](https://mlsanity.com/paper/2603.23178/) (2026)

## Abstract (arXiv)

We present VPN - a content attribution method for recovering provenance information from videos shared online. Platforms, and users, often transform video into different quality, codecs, sizes, shapes, etc. or slightly edit its content such as adding text or emoji, as they are redistributed online. We learn a robust search embedding for matching such video, invariant to these transformations, using full-length or truncated video queries. Once matched against a trusted database of video clips, associated information on the provenance of the clip is presented to the user. We use an inverted index to match temporal chunks of video using late-fusion to combine both visual and audio features. In both cases, features are extracted via a deep neural network trained using contrastive learning on a dataset of original and augmented video clips. We demonstrate high accuracy recall over a corpus of 100,000 videos.