# The 2023 Video Similarity Dataset and Challenge

arXiv: https://arxiv.org/abs/2306.09489 · HTML: https://mlsanity.com/paper/2306.09489/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 51 references that are in the map.
- First posted on arXiv 15 Jun 2023.

## Builds on (most cited first, 24 of 24)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models](https://mlsanity.com/paper/2212.03860/) (2022)
- [YFCC100M: The New Data in Multimedia Research](https://mlsanity.com/paper/1503.01817/) (2015)
- [A Self-Supervised Descriptor for Image Copy Detection](https://mlsanity.com/paper/2202.10261/) (2022)
- [Contextual Transformer Networks for Visual Recognition](https://mlsanity.com/paper/2107.12292/) (2021)
- [The 2021 Image Similarity Dataset and Challenge](https://mlsanity.com/paper/2106.09672/) (2021)
- [ViSiL: Fine-grained Spatio-Temporal Video Similarity Learning](https://mlsanity.com/paper/1908.07410/) (2019)
- [AugLy: Data Augmentations for Robustness](https://mlsanity.com/paper/2201.06494/) (2022)
- [FIVR: Fine-grained Incident Video Retrieval](https://mlsanity.com/paper/1809.04094/) (2018)
- [Contrastive Learning with Large Memory Bank and Negative Embedding Subtraction for Accurate Copy Detection](https://mlsanity.com/paper/2112.04323/) (2021)
- [DnS: Distill-and-Select for Efficient and Accurate Video Indexing and Retrieval](https://mlsanity.com/paper/2106.13266/) (2021)
- [TransVCL: Attention-enhanced Video Copy Localization Network with Flexible Supervision](https://mlsanity.com/paper/2211.13090/) (2022)
- [Video Similarity and Alignment Learning on Partial Video Copy Detection](https://mlsanity.com/paper/2108.01817/) (2021)
- [Self-Supervised Video Similarity Learning](https://mlsanity.com/paper/2304.03378/) (2023)
- [A Large-scale Comprehensive Dataset and Copy-overlap Aware Evaluation Protocol for Segment-level Video Copy Detection](https://mlsanity.com/paper/2203.02654/) (2022)
- [Circulant temporal encoding for video retrieval and temporal alignment](https://mlsanity.com/paper/1506.02588/) (2015)
- [A Dual-level Detection Method for Video Copy Detection](https://mlsanity.com/paper/2305.12361/) (2023)
- [A Similarity Alignment Model for Video Copy Segment Matching](https://mlsanity.com/paper/2305.15679/) (2023)
- [Feature-compatible Progressive Learning for Video Copy Detection](https://mlsanity.com/paper/2304.10305/) (2023)
- [3rd Place Solution to Meta AI Video Similarity Challenge](https://mlsanity.com/paper/2304.11964/) (2023)

## Built on by (most cited first, 9 of 9)

- [The Faiss library](https://mlsanity.com/paper/2401.08281/) (2024)
- [VidProM: A Million-scale Real Prompt-Gallery Dataset for Text-to-Video Diffusion Models](https://mlsanity.com/paper/2403.06098/) (2024)
- [AI-Generated Video Detection via Perceptual Straightening](https://mlsanity.com/paper/2507.00583/) (2025)
- [TIP-I2V: A Million-Scale Real Text and Image Prompt Dataset for Image-to-Video Generation](https://mlsanity.com/paper/2411.04709/) (2024)
- [Counteracting temporal attacks in Video Copy Detection](https://mlsanity.com/paper/2501.11171/) (2025)
- [MatchLM2Lite: A Scalable MLLM-to-Lite Framework for Reproduced Content Identification](https://mlsanity.com/paper/2606.14786/) (2026)
- [EA-Swin: An Embedding-Agnostic Swin Transformer for AI-Generated Video Detection](https://mlsanity.com/paper/2602.17260/) (2026)
- [Tracing Copied Pixels and Regularizing Patch Affinity in Copy Detection](https://mlsanity.com/paper/2602.17484/) (2026)
- [Efficient Logic Gate Networks for Video Copy Detection](https://mlsanity.com/paper/2604.21694/) (2026)

## Abstract (arXiv)

This work introduces a dataset, benchmark, and challenge for the problem of video copy detection and localization. The problem comprises two distinct but related tasks: determining whether a query video shares content with a reference video ("detection"), and additionally temporally localizing the shared content within each video ("localization"). The benchmark is designed to evaluate methods on these two tasks, and simulates a realistic needle-in-haystack setting, where the majority of both query and reference videos are "distractors" containing no copied content. We propose a metric that reflects both detection and localization accuracy. The associated challenge consists of two corresponding tracks, each with restrictions that reflect real-world settings. We provide implementation code for evaluation and baselines. We also analyze the results and methods of the top submissions to the challenge. The dataset, baseline methods and evaluation code is publicly available and will be discussed at a dedicated CVPR'23 workshop.