# Scanning Only Once: An End-to-end Framework for Fast Temporal Grounding in Long Videos

arXiv: https://arxiv.org/abs/2303.08345 · HTML: https://mlsanity.com/paper/2303.08345/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 39 references that are in the map.
- First posted on arXiv 15 Mar 2023.

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Dense-Captioning Events in Videos](https://mlsanity.com/paper/1705.00754/) (2017)
- [TALL: Temporal Activity Localization via Language Query](https://mlsanity.com/paper/1705.02101/) (2017)
- [Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding](https://mlsanity.com/paper/1604.01753/) (2016)
- [Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Language](https://mlsanity.com/paper/1912.03590/) (2019)
- [Span-based Localizing Network for Natural Language Video Localization](https://mlsanity.com/paper/2004.13931/) (2020)
- [TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrieval](https://mlsanity.com/paper/2001.09099/) (2020)
- [A Joint Sequence Fusion Model for Video Question Answering and Retrieval](https://mlsanity.com/paper/1808.02559/) (2018)
- [UnitBox: An Advanced Object Detection Network](https://mlsanity.com/paper/1608.01471/) (2016)
- [Local-Global Video-Text Interactions for Temporal Grounding](https://mlsanity.com/paper/2004.07514/) (2020)
- [Dense Regression Network for Video Grounding](https://mlsanity.com/paper/2004.03545/) (2020)
- [To Find Where You Talk: Temporal Sentence Localization in Video with Attention Based Location Regression](https://mlsanity.com/paper/1804.07014/) (2018)
- [MAN: Moment Alignment Network for Natural Language Moment Retrieval via Iterative Graph Adjustment](https://mlsanity.com/paper/1812.00087/) (2018)
- [Semantic Conditioned Dynamic Modulation for Temporal Sentence Grounding in Videos](https://mlsanity.com/paper/1910.14303/) (2019)
- [MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptions](https://mlsanity.com/paper/2112.00431/) (2021)
- [Temporally Grounding Language Queries in Videos by Contextual Boundary-aware Prediction](https://mlsanity.com/paper/1909.05010/) (2019)
- [Location-aware Graph Convolutional Networks for Video Question Answering](https://mlsanity.com/paper/2008.09105/) (2020)
- [Cross-Modal Interaction Networks for Query-Based Moment Retrieval in Videos](https://mlsanity.com/paper/1906.02497/) (2019)
- [Less is More: Learning Highlight Detection from Video Duration](https://mlsanity.com/paper/1903.00859/) (2019)
- [VLG-Net: Video-Language Graph Matching Network for Video Grounding](https://mlsanity.com/paper/2011.10132/) (2020)
- [Fine-grained Iterative Attention Network for TemporalLanguage Localization in Videos](https://mlsanity.com/paper/2008.02448/) (2020)
- [HANet: Hierarchical Alignment Networks for Video-Text Retrieval](https://mlsanity.com/paper/2107.12059/) (2021)
- [CONE: An Efficient COarse-to-fiNE Alignment Framework for Long Video Temporal Grounding](https://mlsanity.com/paper/2209.10918/) (2022)
- [Progressive Localization Networks for Language-based Moment Localization](https://mlsanity.com/paper/2102.01282/) (2021)
- [Multi-Modal Interaction Graph Convolutional Network for Temporal Language Localization in Videos](https://mlsanity.com/paper/2110.06058/) (2021)

## Built on by (most cited first, 23 of 23)

- [MotionLLM: Understanding Human Behaviors from Human Motions and Videos](https://mlsanity.com/paper/2405.20340/) (2024)
- [SnAG: Scalable and Accurate Video Grounding](https://mlsanity.com/paper/2404.02257/) (2024)
- [Universal Video Temporal Grounding with Generative Multi-modal Large Language Models](https://mlsanity.com/paper/2506.18883/) (2025)
- [SpikeMba: Multi-Modal Spiking Saliency Mamba for Temporal Video Grounding](https://mlsanity.com/paper/2404.01174/) (2024)
- [LinVT: Empower Your Image-level Large Language Model to Understand Videos](https://mlsanity.com/paper/2412.05185/) (2024)
- [ReVisionLLM: Recursive Vision-Language Model for Temporal Grounding in Hour-Long Videos](https://mlsanity.com/paper/2411.14901/) (2024)
- [EAGLE: Egocentric AGgregated Language-video Engine](https://mlsanity.com/paper/2409.17523/) (2024)
- [RGNet: A Unified Clip Retrieval and Grounding Network for Long Videos](https://mlsanity.com/paper/2312.06729/) (2023)
- [SEAL: Semantic Attention Learning for Long Video Representation](https://mlsanity.com/paper/2412.01798/) (2024)
- [Cross-modal Causal Relation Alignment for Video Question Grounding](https://mlsanity.com/paper/2503.07635/) (2025)
- [Object-Shot Enhanced Grounding Network for Egocentric Video](https://mlsanity.com/paper/2505.04270/) (2025)
- [Temporal Sentence Grounding in Streaming Videos](https://mlsanity.com/paper/2308.07102/) (2023)
- [Localizing Events in Videos with Multimodal Queries](https://mlsanity.com/paper/2406.10079/) (2024)
- [Multi-Scale Contrastive Learning for Video Temporal Grounding](https://mlsanity.com/paper/2412.07157/) (2024)
- [DeCafNet: Delegate and Conquer for Efficient Temporal Grounding in Long Videos](https://mlsanity.com/paper/2505.16376/) (2025)
- [Hierarchical Event Memory for Accurate and Low-latency Online Video Temporal Grounding](https://mlsanity.com/paper/2508.04546/) (2025)
- [ArrowGEV: Grounding Events in Video via Learning the Arrow of Time](https://mlsanity.com/paper/2601.06559/) (2026)
- [OmniVTG: A Large-Scale Dataset and Training Paradigm for Open-World Video Temporal Grounding](https://mlsanity.com/paper/2604.25276/) (2026)
- [Temporal-Aware Reasoning Optimization for Video Temporal Grounding](https://mlsanity.com/paper/2606.09248/) (2026)
- [ResidualViT for Efficient Temporally Dense Video Encoding](https://mlsanity.com/paper/2509.13255/) (2025)
- [Point to Span: Zero-Shot Moment Retrieval for Navigating Unseen Hour-Long Videos](https://mlsanity.com/paper/2512.10363/) (2025)
- [HieraMamba: Video Temporal Grounding via Hierarchical Anchor-Mamba Pooling](https://mlsanity.com/paper/2510.23043/) (2025)
- [UniversalVTG: A Universal and Lightweight Foundation Model for Video Temporal Grounding](https://mlsanity.com/paper/2604.08522/) (2026)

## Abstract (arXiv)

Video temporal grounding aims to pinpoint a video segment that matches the query description. Despite the recent advance in short-form videos (textite.g., in minutes), temporal grounding in long videos (textite.g., in hours) is still at its early stage. To address this challenge, a common practice is to employ a sliding window, yet can be inefficient and inflexible due to the limited number of frames within the window. In this work, we propose an end-to-end framework for fast temporal grounding, which is able to model an hours-long video with one-time network execution. Our pipeline is formulated in a coarse-to-fine manner, where we first extract context knowledge from non-overlapped video clips (textiti.e., anchors), and then supplement the anchors that highly response to the query with detailed content knowledge. Besides the remarkably high pipeline efficiency, another advantage of our approach is the capability of capturing long-range temporal correlation, thanks to modeling the entire video as a whole, and hence facilitates more accurate grounding. Experimental results suggest that, on the long-form video datasets MAD and Ego4d, our method significantly outperforms state-of-the-arts, and achieves 14.6× / 102.8× higher efficiency respectively. Project can be found at urlhttps://github.com/afcedf/SOONet.git.