# Correlation-Guided Query-Dependency Calibration for Video Temporal Grounding

arXiv: https://arxiv.org/abs/2311.08835 · HTML: https://mlsanity.com/paper/2311.08835/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 66 of its 93 references that are in the map.
- First posted on arXiv 15 Nov 2023.

## Builds on (most cited first, 30 of 66)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Efficient Streaming Language Models with Attention Sinks](https://mlsanity.com/paper/2309.17453/) (2023)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [CoCa: Contrastive Captioners are Image-Text Foundation Models](https://mlsanity.com/paper/2205.01917/) (2022)
- [Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks](https://mlsanity.com/paper/2004.06165/) (2020)
- [Grounded Language-Image Pre-training](https://mlsanity.com/paper/2112.03857/) (2021)
- [Localizing Moments in Video with Natural Language](https://mlsanity.com/paper/1708.01641/) (2017)
- [FILIP: Fine-grained Interactive Language-Image Pre-Training](https://mlsanity.com/paper/2111.07783/) (2021)
- [TALL: Temporal Activity Localization via Language Query](https://mlsanity.com/paper/1705.02101/) (2017)
- [DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR](https://mlsanity.com/paper/2201.12329/) (2022)
- [Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Language](https://mlsanity.com/paper/1912.03590/) (2019)
- [Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Concepts](https://mlsanity.com/paper/2111.08276/) (2021)
- [Span-based Localizing Network for Natural Language Video Localization](https://mlsanity.com/paper/2004.13931/) (2020)
- [TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrieval](https://mlsanity.com/paper/2001.09099/) (2020)
- [Self-Chained Image-Language Model for Video Localization and Question Answering](https://mlsanity.com/paper/2305.06988/) (2023)
- [UniVTG: Towards Unified Video-Language Temporal Grounding](https://mlsanity.com/paper/2307.16715/) (2023)
- [Dense Regression Network for Video Grounding](https://mlsanity.com/paper/2004.03545/) (2020)
- [Query-Dependent Video Representation for Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2303.13874/) (2023)
- [UMT: Unified Multi-modal Transformers for Joint Video Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2203.12745/) (2022)
- [MAN: Moment Alignment Network for Natural Language Moment Retrieval via Iterative Graph Adjustment](https://mlsanity.com/paper/1812.00087/) (2018)

## Built on by (most cited first, 30 of 31)

- [Universal Video Temporal Grounding with Generative Multi-modal Large Language Models](https://mlsanity.com/paper/2506.18883/) (2025)
- [TimeExpert: An Expert-Guided Video LLM for Video Temporal Grounding](https://mlsanity.com/paper/2508.01699/) (2025)
- [Enrich and Detect: Video Temporal Grounding with Multimodal LLMs](https://mlsanity.com/paper/2510.17023/) (2025)
- [Tempo-R0: A Video-MLLM for Temporal Video Grounding through Efficient Temporal Sensing Reinforcement Learning](https://mlsanity.com/paper/2507.04702/) (2025)
- [AURA: A Fine-Grained Benchmark and Decomposed Metric for Audio-Visual Reasoning](https://mlsanity.com/paper/2508.07470/) (2025)
- [DisTime: Distribution-based Time Representation for Video Large Language Models](https://mlsanity.com/paper/2505.24329/) (2025)
- [Lighthouse: A User-Friendly Library for Reproducible Video Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2408.02901/) (2024)
- [TimeScope: Towards Task-Oriented Temporal Grounding In Long Videos](https://mlsanity.com/paper/2509.26360/) (2025)
- [GraphThinker: Reinforcing Temporally Grounded Video Reasoning with Event Graph Thinking](https://mlsanity.com/paper/2602.17555/) (2026)
- [Empower Words: DualGround for Structured Phrase and Sentence-Level Temporal Grounding](https://mlsanity.com/paper/2510.20244/) (2025)
- [CVA: Context-aware Video-text Alignment for Video Temporal Grounding](https://mlsanity.com/paper/2603.24934/) (2026)
- [GigaCheck: Detecting LLM-generated Content via Object-Centric Span Localization](https://mlsanity.com/paper/2410.23728/) (2024)
- [The Devil is in the Spurious Correlations: Boosting Moment Retrieval with Dynamic Learning](https://mlsanity.com/paper/2501.07305/) (2025)
- [TimeLoc: A Unified End-to-End Framework for Precise Timestamp Localization in Long Videos](https://mlsanity.com/paper/2503.06526/) (2025)
- [How Should Video LLMs Output Time? An Analysis of Efficient Temporal Grounding Paradigms](https://mlsanity.com/paper/2604.08966/) (2026)
- [ActPrompt: In-Domain Feature Adaptation via Action Cues for Video Temporal Grounding](https://mlsanity.com/paper/2408.06622/) (2024)
- [Measure Twice, Cut Once: A Semantic-Oriented Approach to Video Temporal Localization with Video LLMs](https://mlsanity.com/paper/2503.09027/) (2025)
- [SMART: Shot-Aware Multimodal Video Moment Retrieval with Audio-Enhanced MLLM](https://mlsanity.com/paper/2511.14143/) (2025)
- [SlotVTG: Object-Centric Adapter for Generalizable Video Temporal Grounding](https://mlsanity.com/paper/2603.25733/) (2026)
- [EVIDENT: Routing MLLM Adaptation through Entity-Grounded Visual Evidence for Cross-Domain Video Temporal Grounding](https://mlsanity.com/paper/2605.26104/) (2026)
- [SwissADT: An Audio Description Translation System for Swiss Languages](https://mlsanity.com/paper/2411.14967/) (2024)
- [UniversalVTG: A Universal and Lightweight Foundation Model for Video Temporal Grounding](https://mlsanity.com/paper/2604.08522/) (2026)
- [ClipTBP: Clip-Pair based Temporal Boundary Prediction with Boundary-Aware Learning for Moment Retrieval](https://mlsanity.com/paper/2604.27591/) (2026)
- [Multimodal Data Curation Through Ranked Retrieval](https://mlsanity.com/paper/2605.01163/) (2026)
- [Retrieving Any Relevant Moments: Benchmark and Models for Generalized Moment Retrieval](https://mlsanity.com/paper/2605.02623/) (2026)
- [Turing Patterns for Multimedia: Reaction-Diffusion Multi-Modal Fusion for Language-Guided Video Moment Retrieval](https://mlsanity.com/paper/2606.01615/) (2026)
- [SVHighlights: Towards Extremely Long Sport Video Highlight Detection](https://mlsanity.com/paper/2606.06926/) (2026)
- [EventCoT: Event-centric Video Chain-of-thought for Reasoning Temporal Localization](https://mlsanity.com/paper/2607.04872/) (2026)
- [ScanFocus: A Coarse-to-Fine Framework for Spatio-Temporal Video Grounding](https://mlsanity.com/paper/2607.13421/) (2026)
- [REZE: Recognition-Based Zero-Shot Extraction for Video Temporal Grounding](https://mlsanity.com/paper/2608.04480/) (2026)

## Abstract (arXiv)

Temporal Grounding is to identify specific moments or highlights from a video corresponding to textual descriptions. Typical approaches in temporal grounding treat all video clips equally during the encoding process regardless of their semantic relevance with the text query. Therefore, we propose Correlation-Guided DEtection TRansformer (CG-DETR), exploring to provide clues for query-associated video clips within the cross-modal attention. First, we design an adaptive cross-attention with dummy tokens. Dummy tokens conditioned by text query take portions of the attention weights, preventing irrelevant video clips from being represented by the text query. Yet, not all words equally inherit the text query's correlation to video clips. Thus, we further guide the cross-attention map by inferring the fine-grained correlation between video clips and words. We enable this by learning a joint embedding space for high-level concepts, i.e., moment and sentence level, and inferring the clip-word correlation. Lastly, we exploit the moment-specific characteristics and combine them with the context of each video to form a moment-adaptive saliency detector. By exploiting the degrees of text engagement in each video clip, it precisely measures the highlightness of each clip. CG-DETR achieves state-of-the-art results on various benchmarks for temporal grounding. Codes are available at https://github.com/wjun0830/CGDETR.