# Learning Partially-Decorrelated Common Spaces for Ad-hoc Video Search

arXiv: https://arxiv.org/abs/2508.02340 · HTML: https://mlsanity.com/paper/2508.02340/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 66 references that are in the map.
- First posted on arXiv 4 Aug 2025.

## Builds on (most cited first, 28 of 28)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval](https://mlsanity.com/paper/2104.08860/) (2021)
- [Exploring the Limits of Weakly Supervised Pretraining](https://mlsanity.com/paper/1805.00932/) (2018)
- [Multi-modal Transformer for Video Retrieval](https://mlsanity.com/paper/2007.10639/) (2020)
- [X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval](https://mlsanity.com/paper/2207.07285/) (2022)
- [Use What You Have: Video Retrieval Using Representations From Collaborative Experts](https://mlsanity.com/paper/1907.13487/) (2019)
- [A Joint Sequence Fusion Model for Video Question Answering and Retrieval](https://mlsanity.com/paper/1808.02559/) (2018)
- [Fine-grained Video-Text Retrieval with Hierarchical Graph Reasoning](https://mlsanity.com/paper/2003.00392/) (2020)
- [Support-set bottlenecks for video-text representation learning](https://mlsanity.com/paper/2010.02824/) (2020)
- [Large-scale weakly-supervised pre-training for video action recognition](https://mlsanity.com/paper/1905.00561/) (2019)
- [TS2-Net: Token Shift and Selection Transformer for Text-Video Retrieval](https://mlsanity.com/paper/2207.07852/) (2022)
- [Cap4Video: What Can Auxiliary Captions Do for Text-Video Retrieval?](https://mlsanity.com/paper/2301.00184/) (2022)
- [Dual Encoding for Zero-Example Video Retrieval](https://mlsanity.com/paper/1809.06181/) (2018)
- [CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignment](https://mlsanity.com/paper/2209.06430/) (2022)
- [DiffusionRet: Generative Text-Video Retrieval with Diffusion Model](https://mlsanity.com/paper/2303.09867/) (2023)
- [SEA: Sentence Encoder Assembly for Video Retrieval by Textual Queries](https://mlsanity.com/paper/2011.12091/) (2020)
- [DGL: Dynamic Global-Local Prompt Tuning for Text-Video Retrieval](https://mlsanity.com/paper/2401.10588/) (2024)
- [Text-Adaptive Multiple Visual Prototype Matching for Video-Text Retrieval](https://mlsanity.com/paper/2209.13307/) (2022)
- [TRECVID 2019: An Evaluation Campaign to Benchmark Video Activity Detection, Video Captioning and Matching, and Video Search & Retrieval](https://mlsanity.com/paper/2009.09984/) (2020)
- [TRECVID 2020: A comprehensive campaign for evaluating video retrieval tasks across multiple application domains](https://mlsanity.com/paper/2104.13473/) (2021)
- [Towards a Better Tradeoff between Effectiveness and Efficiency in Pre-Ranking: A Learnable Feature Selection based Approach](https://mlsanity.com/paper/2105.07706/) (2021)
- [Dual-Modal Attention-Enhanced Text-Video Retrieval with Triplet Partial Margin Contrastive Learning](https://mlsanity.com/paper/2309.11082/) (2023)
- [An overview on the evaluated video retrieval tasks at TRECVID 2022](https://mlsanity.com/paper/2306.13118/) (2023)
- [Cross-Media Similarity Evaluation for Web Image Retrieval in the Wild](https://mlsanity.com/paper/1709.01305/) (2017)
- [Interpretable Embedding for Ad-hoc Video Search](https://mlsanity.com/paper/2402.11812/) (2024)
- [Improving Interpretable Embeddings for Ad-hoc Video Search with Generative Captions and Multi-word Concept Bank](https://mlsanity.com/paper/2404.06173/) (2024)
- [Renmin University of China at TRECVID 2022: Improving Video Search by Feature Fusion and Negation Understanding](https://mlsanity.com/paper/2211.15039/) (2022)

## Abstract (arXiv)

Ad-hoc Video Search (AVS) involves using a textual query to search for multiple relevant videos in a large collection of unlabeled short videos. The main challenge of AVS is the visual diversity of relevant videos. A simple query such as "Find shots of a man and a woman dancing together indoors" can span a multitude of environments, from brightly lit halls and shadowy bars to dance scenes in black-and-white animations. It is therefore essential to retrieve relevant videos as comprehensively as possible. Current solutions for the AVS task primarily fuse multiple features into one or more common spaces, yet overlook the need for diverse spaces. To fully exploit the expressive capability of individual features, we propose LPD, short for Learning Partially Decorrelated common spaces. LPD incorporates two key innovations: feature-specific common space construction and the de-correlation loss. Specifically, LPD learns a separate common space for each video and text feature, and employs de-correlation loss to diversify the ordering of negative samples across different spaces. To enhance the consistency of multi-space convergence, we designed an entropy-based fair multi-space triplet ranking loss. Extensive experiments on the TRECVID AVS benchmarks (2016-2023) justify the effectiveness of LPD. Moreover, diversity visualizations of LPD's spaces highlight its ability to enhance result diversity.