# Span-based Localizing Network for Natural Language Video Localization

arXiv: https://arxiv.org/abs/2004.13931 · HTML: https://mlsanity.com/paper/2004.13931/

## Key facts

- Cited by 235 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Apr 2020.

## Built on by (most cited first, 30 of 235)

- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [VTimeLLM: Empower LLM to Grasp Video Moments](https://mlsanity.com/paper/2311.18445/) (2023)
- [Egocentric Video-Language Pretraining](https://mlsanity.com/paper/2206.01670/) (2022)
- [VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding](https://mlsanity.com/paper/2403.11481/) (2024)
- [UniVTG: Towards Unified Video-Language Temporal Grounding](https://mlsanity.com/paper/2307.16715/) (2023)
- [Query-Dependent Video Representation for Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2303.13874/) (2023)
- [EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbone](https://mlsanity.com/paper/2307.05463/) (2023)
- [Sequence Parallelism: Long Sequence Training from System Perspective](https://mlsanity.com/paper/2105.13120/) (2021)
- [Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoning](https://mlsanity.com/paper/2402.11435/) (2024)
- [VideoLLM: Modeling Video Sequence with Large Language Models](https://mlsanity.com/paper/2305.13292/) (2023)
- [Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understanding](https://mlsanity.com/paper/2403.09626/) (2024)
- [Animal Kingdom: A Large and Diverse Dataset for Animal Behavior Understanding](https://mlsanity.com/paper/2204.08129/) (2022)
- [TubeDETR: Spatio-Temporal Video Grounding with Transformers](https://mlsanity.com/paper/2203.16434/) (2022)
- [Interventional Video Grounding with Dual Contrastive Learning](https://mlsanity.com/paper/2106.11013/) (2021)
- [Negative Sample Matters: A Renaissance of Metric Learning for Temporal Grounding](https://mlsanity.com/paper/2109.04872/) (2021)
- [Boundary Proposal Network for Two-Stage Natural Language Video Localization](https://mlsanity.com/paper/2103.08109/) (2021)
- [HawkEye: Training Video-Text LLMs for Grounding Text in Videos](https://mlsanity.com/paper/2403.10228/) (2024)
- [Deconfounded Video Moment Retrieval with Causal Intervention](https://mlsanity.com/paper/2106.01534/) (2021)
- [Frame-Voyager: Learning to Query Frames for Video Large Language Models](https://mlsanity.com/paper/2410.03226/) (2024)
- [T*: Re-thinking Temporal Search for Long-Form Video Understanding](https://mlsanity.com/paper/2504.02259/) (2025)
- [Knowing Where to Focus: Event-aware Transformer for Video Grounding](https://mlsanity.com/paper/2308.06947/) (2023)
- [QVHighlights: Detecting Moments and Highlights in Videos via Natural Language Queries](https://mlsanity.com/paper/2107.09609/) (2021)
- [V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoning](https://mlsanity.com/paper/2503.11495/) (2025)
- [UnLoc: A Unified Framework for Video Localization Tasks](https://mlsanity.com/paper/2308.11062/) (2023)
- [Grounded Question-Answering in Long Egocentric Videos](https://mlsanity.com/paper/2312.06505/) (2023)
- [Video Corpus Moment Retrieval with Contrastive Learning](https://mlsanity.com/paper/2105.06247/) (2021)
- [Compositional Temporal Grounding with Structured Variational Cross-Graph Correspondence Learning](https://mlsanity.com/paper/2203.13049/) (2022)
- [MomentDiff: Generative Video Moment Retrieval from Random to Real](https://mlsanity.com/paper/2307.02869/) (2023)
- [Natural Language Video Localization: A Revisit in Span-based Question Answering Framework](https://mlsanity.com/paper/2102.13558/) (2021)
- [On Pursuit of Designing Multi-modal Transformer for Video Grounding](https://mlsanity.com/paper/2109.06085/) (2021)

## Abstract (arXiv)

Given an untrimmed video and a text query, natural language video localization (NLVL) is to locate a matching span from the video that semantically corresponds to the query. Existing solutions formulate NLVL either as a ranking task and apply multimodal matching architecture, or as a regression task to directly regress the target video span. In this work, we address NLVL task with a span-based QA approach by treating the input video as text passage. We propose a video span localizing network (VSLNet), on top of the standard span-based QA framework, to address NLVL. The proposed VSLNet tackles the differences between NLVL and span-based QA through a simple yet effective query-guided highlighting (QGH) strategy. The QGH guides VSLNet to search for matching video span within a highlighted region. Through extensive experiments on three benchmark datasets, we show that the proposed VSLNet outperforms the state-of-the-art methods; and adopting span-based QA framework is a promising direction to solve NLVL.