# GMMFormer v2: An Uncertainty-aware Framework for Partially Relevant Video Retrieval

arXiv: https://arxiv.org/abs/2405.13824 · HTML: https://mlsanity.com/paper/2405.13824/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 49 references that are in the map.
- First posted on arXiv 22 May 2024.

## Builds on (most cited first, 17 of 17)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation](https://mlsanity.com/paper/2307.06942/) (2023)
- [InternVideo2: Scaling Foundation Models for Multimodal Video Understanding](https://mlsanity.com/paper/2403.15377/) (2024)
- [Use What You Have: Video Retrieval Using Representations From Collaborative Experts](https://mlsanity.com/paper/1907.13487/) (2019)
- [MomentDiff: Generative Video Moment Retrieval from Random to Real](https://mlsanity.com/paper/2307.02869/) (2023)
- [Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representations](https://mlsanity.com/paper/2211.11427/) (2022)
- [UATVR: Uncertainty-Adaptive Text-Video Retrieval](https://mlsanity.com/paper/2301.06309/) (2023)
- [Modeling Uncertainty with Hedged Instance Embedding](https://mlsanity.com/paper/1810.00319/) (2018)
- [DiffusionRet: Generative Text-Video Retrieval with Diffusion Model](https://mlsanity.com/paper/2303.09867/) (2023)
- [Text-Video Retrieval with Disentangled Conceptualization and Set-to-Set Alignment](https://mlsanity.com/paper/2305.12218/) (2023)
- [Unleashing Unlabeled Data: A Paradigm for Cross-View Geo-Localization](https://mlsanity.com/paper/2403.14198/) (2024)
- [A Hierarchical Multi-Modal Encoder for Moment Localization in Video Corpus](https://mlsanity.com/paper/2011.09046/) (2020)
- [TENET: Transformer Encoding Network for Effective Temporal Flow on Motion Prediction](https://mlsanity.com/paper/2207.00170/) (2022)
- [TVTSv2: Learning Out-of-the-box Spatiotemporal Visual Representations at Scale](https://mlsanity.com/paper/2305.14173/) (2023)
- [TG-VQA: Ternary Game of Video Question Answering](https://mlsanity.com/paper/2305.10049/) (2023)

## Built on by (most cited first, 12 of 12)

- [Towards Efficient Partially Relevant Video Retrieval with Active Moment Discovering](https://mlsanity.com/paper/2504.10920/) (2025)
- [HLFormer: Enhancing Partially Relevant Video Retrieval with Hyperbolic Learning](https://mlsanity.com/paper/2507.17402/) (2025)
- [Mitigating Semantic Collapse in Partially Relevant Video Retrieval](https://mlsanity.com/paper/2510.27432/) (2025)
- [Uneven Event Modeling for Partially Relevant Video Retrieval](https://mlsanity.com/paper/2506.00891/) (2025)
- [Enhancing Partially Relevant Video Retrieval with Robust Alignment Learning](https://mlsanity.com/paper/2509.01383/) (2025)
- [Imagine Before Concentration: Diffusion-Guided Registers Enhance Partially Relevant Video Retrieval](https://mlsanity.com/paper/2604.03653/) (2026)
- [Revisiting Uncertainty: On Evidential Learning for Partially Relevant Video Retrieval](https://mlsanity.com/paper/2605.06083/) (2026)
- [ProPy: Building Interactive Prompt Pyramids upon CLIP for Partially Relevant Video Retrieval](https://mlsanity.com/paper/2508.19024/) (2025)
- [Knowledge-Refined Dual Context-Aware Network for Partially Relevant Video Retrieval](https://mlsanity.com/paper/2603.23902/) (2026)
- [Intrinsic Temporal Adaptation of CLIP for Partially Relevant Video Retrieval](https://mlsanity.com/paper/2609.04800/) (2026)
- [CurvSpec: Adaptive Multi-Curvature Learning for Partial Relevant Video Retrieval](https://mlsanity.com/paper/2609.36815/) (2026)
- [TF-PRVR: Training-Free Partially Relevant Video Retrieval](https://mlsanity.com/paper/2610.07925/) (2026)

## Abstract (arXiv)

Given a text query, partially relevant video retrieval (PRVR) aims to retrieve untrimmed videos containing relevant moments. Due to the lack of moment annotations, the uncertainty lying in clip modeling and text-clip correspondence leads to major challenges. Despite the great progress, existing solutions either sacrifice efficiency or efficacy to capture varying and uncertain video moments. What's worse, few methods have paid attention to the text-clip matching pattern under such uncertainty, exposing the risk of semantic collapse. To address these issues, we present GMMFormer v2, an uncertainty-aware framework for PRVR. For clip modeling, we improve a strong baseline GMMFormer with a novel temporal consolidation module upon multi-scale contextual features, which maintains efficiency and improves the perception for varying moments. To achieve uncertainty-aware text-clip matching, we upgrade the query diverse loss in GMMFormer to facilitate fine-grained uniformity and propose a novel optimal matching loss for fine-grained text-clip alignment. Their collaboration alleviates the semantic collapse phenomenon and neatly promotes accurate correspondence between texts and moments. We conduct extensive experiments and ablation studies on three PRVR benchmarks, demonstrating remarkable improvement of GMMFormer v2 compared to the past SOTA competitor and the versatility of uncertainty-aware text-clip matching for PRVR. Code is available at urlhttps://github.com/huangmozhi9527/GMMFormerv2.