# Parallel Attention Network with Sequence Matching for Video Grounding

arXiv: https://arxiv.org/abs/2105.08481 · HTML: https://mlsanity.com/paper/2105.08481/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 78 references that are in the map.
- First posted on arXiv 18 May 2021.

## Builds on (most cited first, 30 of 54)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [VL-BERT: Pre-training of Generic Visual-Linguistic Representations](https://mlsanity.com/paper/1908.08530/) (2019)
- [Dense-Captioning Events in Videos](https://mlsanity.com/paper/1705.00754/) (2017)
- [Localizing Moments in Video with Natural Language](https://mlsanity.com/paper/1708.01641/) (2017)
- [TALL: Temporal Activity Localization via Language Query](https://mlsanity.com/paper/1705.02101/) (2017)
- [ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering](https://mlsanity.com/paper/1906.02467/) (2019)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding](https://mlsanity.com/paper/1604.01753/) (2016)
- [HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training](https://mlsanity.com/paper/2005.00200/) (2020)
- [End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF](https://mlsanity.com/paper/1603.01354/) (2016)
- [Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Language](https://mlsanity.com/paper/1912.03590/) (2019)
- [Bidirectional Attention Flow for Machine Comprehension](https://mlsanity.com/paper/1611.01603/) (2016)
- [BSN: Boundary Sensitive Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1806.02964/) (2018)
- [ActBERT: Learning Global-Local Video-Text Representations](https://mlsanity.com/paper/2011.07231/) (2020)
- [Searching for A Robust Neural Architecture in Four GPU Hours](https://mlsanity.com/paper/1910.04465/) (2019)
- [Span-based Localizing Network for Natural Language Video Localization](https://mlsanity.com/paper/2004.13931/) (2020)
- [TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrieval](https://mlsanity.com/paper/2001.09099/) (2020)
- [Local-Global Video-Text Interactions for Temporal Grounding](https://mlsanity.com/paper/2004.07514/) (2020)
- [Dense Regression Network for Video Grounding](https://mlsanity.com/paper/2004.03545/) (2020)

## Built on by (most cited first, 22 of 22)

- [Temporal Sentence Grounding in Videos: A Survey and Future Directions](https://mlsanity.com/paper/2201.08071/) (2022)
- [SnAG: Scalable and Accurate Video Grounding](https://mlsanity.com/paper/2404.02257/) (2024)
- [Towards Balanced Alignment: Modal-Enhanced Semantic Modeling for Video Moment Retrieval](https://mlsanity.com/paper/2312.12155/) (2023)
- [Skimming, Locating, then Perusing: A Human-Like Framework for Natural Language Video Localization](https://mlsanity.com/paper/2207.13450/) (2022)
- [End-to-End Modeling via Information Tree for One-Shot Natural Language Spatial Video Grounding](https://mlsanity.com/paper/2203.08013/) (2022)
- [Rethinking the Video Sampling and Reasoning Strategies for Temporal Sentence Grounding](https://mlsanity.com/paper/2301.00514/) (2023)
- [Background-aware Moment Detection for Video Moment Retrieval](https://mlsanity.com/paper/2306.02728/) (2023)
- [Towards Debiasing Temporal Sentence Grounding in Video](https://mlsanity.com/paper/2111.04321/) (2021)
- [Entity-aware and Motion-aware Transformers for Language-driven Action Localization in Videos](https://mlsanity.com/paper/2205.05854/) (2022)
- [MS-DETR: Natural Language Video Localization with Sampling Moment-Moment Interaction](https://mlsanity.com/paper/2305.18969/) (2023)
- [Temporal Sentence Grounding in Streaming Videos](https://mlsanity.com/paper/2308.07102/) (2023)
- [Bias-Conflict Sample Synthesis and Adversarial Removal Debias Strategy for Temporal Sentence Grounding in Video](https://mlsanity.com/paper/2401.07567/) (2024)
- [MRTNet: Multi-Resolution Temporal Network for Video Sentence Grounding](https://mlsanity.com/paper/2212.13163/) (2022)
- [Temporally Grounding Instructional Diagrams in Unconstrained Videos](https://mlsanity.com/paper/2407.12066/) (2024)
- [Efficient Temporal Sentence Grounding in Videos with Multi-Teacher Knowledge Distillation](https://mlsanity.com/paper/2308.03725/) (2023)
- [MLP: Motion Label Prior for Temporal Sentence Localization in Untrimmed 3D Human Motions](https://mlsanity.com/paper/2404.13657/) (2024)
- [TimeLoc: A Unified End-to-End Framework for Precise Timestamp Localization in Long Videos](https://mlsanity.com/paper/2503.06526/) (2025)
- [Generative Video Diffusion for Unseen Novel Semantic Video Moment Retrieval](https://mlsanity.com/paper/2401.13329/) (2024)
- [Audio Does Matter: Importance-Aware Multi-Granularity Fusion for Video Moment Retrieval](https://mlsanity.com/paper/2508.04273/) (2025)
- [Coarse to Fine: Video Retrieval before Moment Localization](https://mlsanity.com/paper/2110.07201/) (2021)
- [Prompt When the Animal is: Temporal Animal Behavior Grounding with Positional Recovery Training](https://mlsanity.com/paper/2405.05523/) (2024)
- [OVG-HQ: Online Video Grounding with Hybrid-modal Queries](https://mlsanity.com/paper/2508.11903/) (2025)

## Abstract (arXiv)

Given a video, video grounding aims to retrieve a temporal moment that semantically corresponds to a language query. In this work, we propose a Parallel Attention Network with Sequence matching (SeqPAN) to address the challenges in this task: multi-modal representation learning, and target moment boundary prediction. We design a self-guided parallel attention module to effectively capture self-modal contexts and cross-modal attentive information between video and text. Inspired by sequence labeling tasks in natural language processing, we split the ground truth moment into begin, inside, and end regions. We then propose a sequence matching strategy to guide start/end boundary predictions using region labels. Experimental results on three datasets show that SeqPAN is superior to state-of-the-art methods. Furthermore, the effectiveness of the self-guided parallel attention module and the sequence matching module is verified.