# EgoCVR: An Egocentric Benchmark for Fine-Grained Composed Video Retrieval

arXiv: https://arxiv.org/abs/2407.16658 · HTML: https://mlsanity.com/paper/2407.16658/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 59 references that are in the map.
- First posted on arXiv 23 Jul 2024.

## Builds on (most cited first, 30 of 51)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval](https://mlsanity.com/paper/2104.00650/) (2021)
- [ImageBind: One Embedding Space To Bind Them All](https://mlsanity.com/paper/2305.05665/) (2023)
- [HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips](https://mlsanity.com/paper/1906.03327/) (2019)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval](https://mlsanity.com/paper/2104.08860/) (2021)
- [DataComp: In search of the next generation of multimodal datasets](https://mlsanity.com/paper/2304.14108/) (2023)
- [Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives](https://mlsanity.com/paper/2311.18259/) (2023)
- [VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research](https://mlsanity.com/paper/1904.03493/) (2019)
- [LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment](https://mlsanity.com/paper/2310.01852/) (2023)
- [Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Models](https://mlsanity.com/paper/2108.04024/) (2021)
- [Composing Text and Image for Image Retrieval - An Empirical Odyssey](https://mlsanity.com/paper/1812.07119/) (2018)
- [Egocentric Video-Language Pretraining](https://mlsanity.com/paper/2206.01670/) (2022)
- [Learning Video Representations from Large Language Models](https://mlsanity.com/paper/2212.04501/) (2022)
- [Zero-Shot Composed Image Retrieval with Textual Inversion](https://mlsanity.com/paper/2303.15247/) (2023)
- [VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset](https://mlsanity.com/paper/2305.18500/) (2023)
- [Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval](https://mlsanity.com/paper/2302.03084/) (2023)
- [mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video](https://mlsanity.com/paper/2302.00402/) (2023)
- [Automatic Spatially-aware Fashion Concept Discovery](https://mlsanity.com/paper/1708.01311/) (2017)
- [EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbone](https://mlsanity.com/paper/2307.05463/) (2023)

## Built on by (most cited first, 21 of 21)

- [FineCIR: Explicit Parsing of Fine-Grained Modification Semantics for Composed Image Retrieval](https://mlsanity.com/paper/2503.21309/) (2025)
- [Reason-before-Retrieve: One-Stage Reflective Chain-of-Thoughts for Training-Free Zero-Shot Composed Image Retrieval](https://mlsanity.com/paper/2412.11077/) (2024)
- [HUD: Hierarchical Uncertainty-Aware Disambiguation Network for Composed Video Retrieval](https://mlsanity.com/paper/2512.02792/) (2025)
- [Modality Curation: Building Universal Embeddings for Advanced Multimodal Information Retrieval](https://mlsanity.com/paper/2505.19650/) (2025)
- [Challenges and Trends in Egocentric Vision: A Survey](https://mlsanity.com/paper/2503.15275/) (2025)
- [A Comprehensive Survey on Composed Image Retrieval](https://mlsanity.com/paper/2502.18495/) (2025)
- [Towards Universal Video Retrieval: Generalizing Video Embedding via Synthesized Multimodal Pyramid Curriculum](https://mlsanity.com/paper/2510.27571/) (2025)
- [Beyond Simple Edits: Composed Video Retrieval with Dense Modifications](https://mlsanity.com/paper/2508.14039/) (2025)
- [From Play to Replay: Composed Video Retrieval for Temporally Fine-Grained Videos](https://mlsanity.com/paper/2506.05274/) (2025)
- [ConViS-Bench: Estimating Video Similarity Through Semantic Concepts](https://mlsanity.com/paper/2509.19245/) (2025)
- [VeRVE: Versatile Retrieval for Videos via Unified Embeddings](https://mlsanity.com/paper/2601.12193/) (2026)
- [Composed Multi-modal Retrieval: A Survey of Approaches and Applications](https://mlsanity.com/paper/2503.01334/) (2025)
- [CoVA: Text-Guided Composed Video Retrieval for Audio-Visual Content](https://mlsanity.com/paper/2601.22508/) (2026)
- [Beyond Language: Grounding Referring Expressions with Hand Pointing in Egocentric Vision](https://mlsanity.com/paper/2603.26646/) (2026)
- [G-MIXER: Geodesic Mixup-based Implicit Semantic Expansion and Explicit Semantic Re-ranking for Zero-Shot Composed Image Retrieval](https://mlsanity.com/paper/2604.14710/) (2026)
- [FIGROTD: A Friendly-to-Handle Dataset for Image Guided Retrieval with Optional Text](https://mlsanity.com/paper/2511.22247/) (2025)
- [PREGEN: Uncovering Latent Thoughts in Composed Video Retrieval](https://mlsanity.com/paper/2601.13797/) (2026)
- [X-Aligner: Composed Visual Retrieval without the Bells and Whistles](https://mlsanity.com/paper/2601.16582/) (2026)
- [Learning to Evolve Scenes: Reasoning about Human Activities with Scene Graphs](https://mlsanity.com/paper/2607.02425/) (2026)
- [Learning Sample-wise Rank-aware Interpolation Weights for Composed Visual Data Retrieval](https://mlsanity.com/paper/2608.22500/) (2026)
- [Beyond Similarity: Foundation Models as an Efficient Backbone for Training-Free Composed Video Retrieval](https://mlsanity.com/paper/2609.10008/) (2026)

## Abstract (arXiv)

In Composed Video Retrieval, a video and a textual description which modifies the video content are provided as inputs to the model. The aim is to retrieve the relevant video with the modified content from a database of videos. In this challenging task, the first step is to acquire large-scale training datasets and collect high-quality benchmarks for evaluation. In this work, we introduce EgoCVR, a new evaluation benchmark for fine-grained Composed Video Retrieval using large-scale egocentric video datasets. EgoCVR consists of 2,295 queries that specifically focus on high-quality temporal video understanding. We find that existing Composed Video Retrieval frameworks do not achieve the necessary high-quality temporal video understanding for this task. To address this shortcoming, we adapt a simple training-free method, propose a generic re-ranking framework for Composed Video Retrieval, and demonstrate that this achieves strong results on EgoCVR. Our code and benchmark are freely available at https://github.com/ExplainableML/EgoCVR.