# Video Similarity and Alignment Learning on Partial Video Copy Detection

arXiv: https://arxiv.org/abs/2108.01817 · HTML: https://mlsanity.com/paper/2108.01817/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 27 references that are in the map.
- First posted on arXiv 4 Aug 2021.

## Builds on (most cited first, 5 of 5)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [ViSiL: Fine-grained Spatio-Temporal Video Similarity Learning](https://mlsanity.com/paper/1908.07410/) (2019)
- [FIVR: Fine-grained Incident Video Retrieval](https://mlsanity.com/paper/1809.04094/) (2018)
- [Self-supervised Video Retrieval Transformer Network](https://mlsanity.com/paper/2104.07993/) (2021)
- [A Fast Partial Video Copy Detection Using KNN and Global Feature Database](https://mlsanity.com/paper/2105.01713/) (2021)

## Built on by (most cited first, 12 of 12)

- [TransVCL: Attention-enhanced Video Copy Localization Network with Flexible Supervision](https://mlsanity.com/paper/2211.13090/) (2022)
- [Self-Supervised Video Similarity Learning](https://mlsanity.com/paper/2304.03378/) (2023)
- [A Large-scale Comprehensive Dataset and Copy-overlap Aware Evaluation Protocol for Segment-level Video Copy Detection](https://mlsanity.com/paper/2203.02654/) (2022)
- [The 2023 Video Similarity Dataset and Challenge](https://mlsanity.com/paper/2306.09489/) (2023)
- [VADER: Video Alignment Differencing and Retrieval](https://mlsanity.com/paper/2303.13193/) (2023)
- [Sync from the Sea: Retrieving Alignable Videos from Large-Scale Datasets](https://mlsanity.com/paper/2409.01445/) (2024)
- [STAR-GNN: Spatial-Temporal Video Representation for Content-based Retrieval](https://mlsanity.com/paper/2208.06966/) (2022)
- [Tracing Copied Pixels and Regularizing Patch Affinity in Copy Detection](https://mlsanity.com/paper/2602.17484/) (2026)
- [VSCD: Video-based Scene Change Detection in Unaligned Scenes](https://mlsanity.com/paper/2605.20821/) (2026)
- [Anomalous Frame Detection by Grouping Frame Similarities between Two Videos Computed by Vision-Language Model to Extract Expert Workers' Unique Actions](https://mlsanity.com/paper/2607.10598/) (2026)
- [Anomalous Frame Detection Using VLM-Based Description Comparison for Extracting Expert-Specific Actions and Contextual Decision-Making Scenes with Intra-Video Self-Similarity](https://mlsanity.com/paper/2607.11957/) (2026)
- [Generative Uncertainty as a Self-supervised Signal for Semantic Similarity Learning](https://mlsanity.com/paper/2609.35341/) (2026)

## Abstract (arXiv)

Existing video copy detection methods generally measure video similarity based on spatial similarities between key frames, neglecting the latent similarity in temporal dimension, so that the video similarity is biased towards spatial information. There are methods modeling unified video similarity in an end-to-end way, but losing detailed partial alignment information, which causes the incapability of copy segments localization. To address the above issues, we propose the Video Similarity and Alignment Learning (VSAL) approach, which jointly models spatial similarity, temporal similarity and partial alignment. To mitigate the spatial similarity bias, we model the temporal similarity as the mask map predicted from frame-level spatial similarity, where each element indicates the probability of frame pair lying right on the partial alignments. To further localize partial copies, the step map is learned from the spatial similarity where the elements indicate extending directions of the current partial alignments on the spatial-temporal similarity map. Obtained from the mask map, the start points extend out into partial optimal alignments following instructions of the step map. With the similarity and alignment learning strategy, VSAL achieves the state-of-the-art F1-score on VCDB core dataset. Furthermore, we construct a new benchmark of partial video copy detection and localization by adding new segment-level annotations for FIVR-200k dataset, where VSAL also achieves the best performance, verifying its effectiveness in more challenging situations. Our project is publicly available at https://pvcd-vsal.github.io/vsal/.