# CONE: An Efficient COarse-to-fiNE Alignment Framework for Long Video Temporal Grounding

arXiv: https://arxiv.org/abs/2209.10918 · HTML: https://mlsanity.com/paper/2209.10918/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 53 references that are in the map.
- First posted on arXiv 22 Sep 2022.

## Builds on (most cited first, 30 of 39)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [CLIP-Adapter: Better Vision-Language Models with Feature Adapters](https://mlsanity.com/paper/2110.04544/) (2021)
- [Self-Supervised Learning of Pretext-Invariant Representations](https://mlsanity.com/paper/1912.01991/) (2019)
- [Dense-Captioning Events in Videos](https://mlsanity.com/paper/1705.00754/) (2017)
- [Localizing Moments in Video with Natural Language](https://mlsanity.com/paper/1708.01641/) (2017)
- [CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval](https://mlsanity.com/paper/2104.08860/) (2021)
- [VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding](https://mlsanity.com/paper/2109.14084/) (2021)
- [TALL: Temporal Activity Localization via Language Query](https://mlsanity.com/paper/1705.02101/) (2017)
- [TVQA: Localized, Compositional Video Question Answering](https://mlsanity.com/paper/1809.01696/) (2018)
- [Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Language](https://mlsanity.com/paper/1912.03590/) (2019)
- [Long-Term Feature Banks for Detailed Video Understanding](https://mlsanity.com/paper/1812.05038/) (2018)
- [Span-based Localizing Network for Natural Language Video Localization](https://mlsanity.com/paper/2004.13931/) (2020)
- [Egocentric Video-Language Pretraining](https://mlsanity.com/paper/2206.01670/) (2022)
- [TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrieval](https://mlsanity.com/paper/2001.09099/) (2020)
- [Movie Description](https://mlsanity.com/paper/1605.03705/) (2016)
- [Dense Regression Network for Video Grounding](https://mlsanity.com/paper/2004.03545/) (2020)
- [To Find Where You Talk: Temporal Sentence Localization in Video with Attention Based Location Regression](https://mlsanity.com/paper/1804.07014/) (2018)
- [TVQA+: Spatio-Temporal Grounding for Video Question Answering](https://mlsanity.com/paper/1904.11574/) (2019)
- [Towards Long-Form Video Understanding](https://mlsanity.com/paper/2106.11310/) (2021)
- [Multilevel Language and Vision Integration for Text-to-Clip Retrieval](https://mlsanity.com/paper/1804.05113/) (2018)
- [COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining](https://mlsanity.com/paper/2102.08473/) (2021)
- [MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptions](https://mlsanity.com/paper/2112.00431/) (2021)
- [Long Movie Clip Classification with State-Space Video Models](https://mlsanity.com/paper/2204.01692/) (2022)
- [TubeDETR: Spatio-Temporal Video Grounding with Transformers](https://mlsanity.com/paper/2203.16434/) (2022)
- [Interventional Video Grounding with Dual Contrastive Learning](https://mlsanity.com/paper/2106.11013/) (2021)
- [ExCL: Extractive Clip Localization Using Natural Language Descriptions](https://mlsanity.com/paper/1904.02755/) (2019)
- [TALLFormer: Temporal Action Localization with a Long-memory Transformer](https://mlsanity.com/paper/2204.01680/) (2022)

## Built on by (most cited first, 27 of 27)

- [AssistGPT: A General Multi-modal Assistant that can Plan, Execute, Inspect, and Learn](https://mlsanity.com/paper/2306.08640/) (2023)
- [SnAG: Scalable and Accurate Video Grounding](https://mlsanity.com/paper/2404.02257/) (2024)
- [Universal Video Temporal Grounding with Generative Multi-modal Large Language Models](https://mlsanity.com/paper/2506.18883/) (2025)
- [Scanning Only Once: An End-to-end Framework for Fast Temporal Grounding in Long Videos](https://mlsanity.com/paper/2303.08345/) (2023)
- [ReVisionLLM: Recursive Vision-Language Model for Temporal Grounding in Hour-Long Videos](https://mlsanity.com/paper/2411.14901/) (2024)
- [RGNet: A Unified Clip Retrieval and Grounding Network for Long Videos](https://mlsanity.com/paper/2312.06729/) (2023)
- [SEAL: Semantic Attention Learning for Long Video Representation](https://mlsanity.com/paper/2412.01798/) (2024)
- [Object-Shot Enhanced Grounding Network for Egocentric Video](https://mlsanity.com/paper/2505.04270/) (2025)
- [Boundary-Denoising for Video Activity Localization](https://mlsanity.com/paper/2304.02934/) (2023)
- [ObjectNLQ @ Ego4D Episodic Memory Challenge 2024](https://mlsanity.com/paper/2406.15778/) (2024)
- [MINOTAUR: Multi-task Video Grounding From Multimodal Queries](https://mlsanity.com/paper/2302.08063/) (2023)
- [Localizing Events in Videos with Multimodal Queries](https://mlsanity.com/paper/2406.10079/) (2024)
- [Multi-Scale Contrastive Learning for Video Temporal Grounding](https://mlsanity.com/paper/2412.07157/) (2024)
- [DeCafNet: Delegate and Conquer for Efficient Temporal Grounding in Long Videos](https://mlsanity.com/paper/2505.16376/) (2025)
- [An Efficient COarse-to-fiNE Alignment Framework @ Ego4D Natural Language Queries Challenge 2022](https://mlsanity.com/paper/2211.08776/) (2022)
- [SynopGround: A Large-Scale Dataset for Multi-Paragraph Video Grounding from TV Dramas and Synopses](https://mlsanity.com/paper/2408.01669/) (2024)
- [ArrowGEV: Grounding Events in Video via Learning the Arrow of Time](https://mlsanity.com/paper/2601.06559/) (2026)
- [OmniVTG: A Large-Scale Dataset and Training Paradigm for Open-World Video Temporal Grounding](https://mlsanity.com/paper/2604.25276/) (2026)
- [Learning Temporal Sentence Grounding From Narrated EgoVideos](https://mlsanity.com/paper/2310.17395/) (2023)
- [SHINE: Saliency-aware HIerarchical NEgative Ranking for Compositional Temporal Grounding](https://mlsanity.com/paper/2407.05118/) (2024)
- [ShotFinder: Imagination-Driven Open-Domain Video Shot Retrieval via Web Search](https://mlsanity.com/paper/2601.23232/) (2026)
- [ResidualViT for Efficient Temporally Dense Video Encoding](https://mlsanity.com/paper/2509.13255/) (2025)
- [Point to Span: Zero-Shot Moment Retrieval for Navigating Unseen Hour-Long Videos](https://mlsanity.com/paper/2512.10363/) (2025)
- [Infusing Environmental Captions for Long-Form Video Language Grounding](https://mlsanity.com/paper/2408.02336/) (2024)
- [Uncertainty-quantified Rollout Policy Adaptation for Unlabelled Cross-domain Temporal Grounding](https://mlsanity.com/paper/2508.06317/) (2025)
- [HieraMamba: Video Temporal Grounding via Hierarchical Anchor-Mamba Pooling](https://mlsanity.com/paper/2510.23043/) (2025)
- [UniversalVTG: A Universal and Lightweight Foundation Model for Video Temporal Grounding](https://mlsanity.com/paper/2604.08522/) (2026)

## Abstract (arXiv)

This paper tackles an emerging and challenging problem of long video temporal grounding~(VTG) that localizes video moments related to a natural language (NL) query. Compared with short videos, long videos are also highly demanded but less explored, which brings new challenges in higher inference computation cost and weaker multi-modal alignment. To address these challenges, we propose CONE, an efficient COarse-to-fiNE alignment framework. CONE is a plug-and-play framework on top of existing VTG models to handle long videos through a sliding window mechanism. Specifically, CONE (1) introduces a query-guided window selection strategy to speed up inference, and (2) proposes a coarse-to-fine mechanism via a novel incorporation of contrastive learning to enhance multi-modal alignment for long videos. Extensive experiments on two large-scale long VTG benchmarks consistently show both substantial performance gains (e.g., from 3.13% to 6.87% on MAD) and state-of-the-art results. Analyses also reveal higher efficiency as the query-guided window selection mechanism accelerates inference time by 2x on Ego4D-NLQ and 15x on MAD while keeping SOTA results. Codes have been released at https://github.com/houzhijian/CONE.