# Multi-Scale 2D Temporal Adjacent Networks for Moment Localization with Natural Language

arXiv: https://arxiv.org/abs/2012.02646 · HTML: https://mlsanity.com/paper/2012.02646/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Dec 2020.

## Built on by (most cited first, 29 of 29)

- [Time-R1: Post-Training Large Vision Language Model for Temporal Video Grounding](https://mlsanity.com/paper/2503.13377/) (2025)
- [Negative Sample Matters: A Renaissance of Metric Learning for Temporal Grounding](https://mlsanity.com/paper/2109.04872/) (2021)
- [Towards Visual-Prompt Temporal Answering Grounding in Medical Instructional Video](https://mlsanity.com/paper/2203.06667/) (2022)
- [Towards Surveillance Video-and-Language Understanding: New Dataset, Baselines, and Challenges](https://mlsanity.com/paper/2309.13925/) (2023)
- [Temporal Sentence Grounding in Videos: A Survey and Future Directions](https://mlsanity.com/paper/2201.08071/) (2022)
- [Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMs](https://mlsanity.com/paper/2308.13812/) (2023)
- [MUGEN: A Playground for Video-Audio-Text Multimodal Understanding and GENeration](https://mlsanity.com/paper/2204.08058/) (2022)
- [ReLER@ZJU-Alibaba Submission to the Ego4D Natural Language Queries Challenge 2022](https://mlsanity.com/paper/2207.00383/) (2022)
- [A Survey on Video Moment Localization](https://mlsanity.com/paper/2306.07515/) (2023)
- [TempSamp-R1: Effective Temporal Sampling with Reinforcement Fine-Tuning for Video LLMs](https://mlsanity.com/paper/2509.18056/) (2025)
- [Unified Static and Dynamic Network: Efficient Temporal Filtering for Video Grounding](https://mlsanity.com/paper/2403.14174/) (2024)
- [Enrich and Detect: Video Temporal Grounding with Multimodal LLMs](https://mlsanity.com/paper/2510.17023/) (2025)
- [Siamese Learning with Joint Alignment and Regression for Weakly-Supervised Video Paragraph Grounding](https://mlsanity.com/paper/2403.11463/) (2024)
- [VideoExpert: Augmented LLM for Temporal-Sensitive Video Understanding](https://mlsanity.com/paper/2504.07519/) (2025)
- [MS-DETR: Natural Language Video Localization with Sampling Moment-Moment Interaction](https://mlsanity.com/paper/2305.18969/) (2023)
- [Temporal Sentence Grounding in Streaming Videos](https://mlsanity.com/paper/2308.07102/) (2023)
- [Towards Parameter-Efficient Integration of Pre-Trained Language Models In Temporal Video Grounding](https://mlsanity.com/paper/2209.13359/) (2022)
- [Soft-Landing Strategy for Alleviating the Task Discrepancy Problem in Temporal Action Localization Tasks](https://mlsanity.com/paper/2211.06023/) (2022)
- [Annotations as Rollouts: Efficient and Scalable Reinforcement Learning for Video MLLMs](https://mlsanity.com/paper/2608.20492/) (2026)
- [SHINE: Saliency-aware HIerarchical NEgative Ranking for Compositional Temporal Grounding](https://mlsanity.com/paper/2407.05118/) (2024)
- [DiffusionVMR: Diffusion Model for Joint Video Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2308.15109/) (2023)
- [Exploiting Feature Diversity for Make-up Temporal Video Grounding](https://mlsanity.com/paper/2208.06179/) (2022)
- [No-frills Temporal Video Grounding: Multi-Scale Neighboring Attention and Zoom-in Boundary Detection](https://mlsanity.com/paper/2307.10567/) (2023)
- [Disentangle and denoise: Tackling context misalignment for video moment retrieval](https://mlsanity.com/paper/2408.07600/) (2024)
- [Consistency of Compositional Generalization across Multiple Levels](https://mlsanity.com/paper/2412.13636/) (2024)
- [An empirical study of the effect of video encoders on Temporal Video Grounding](https://mlsanity.com/paper/2510.17007/) (2025)
- [Generation-Guided Multi-Level Unified Network for Video Grounding](https://mlsanity.com/paper/2303.07748/) (2023)
- [Multi-scale 2D Temporal Map Diffusion Models for Natural Language Video Localization](https://mlsanity.com/paper/2401.08232/) (2024)
- [A Paradigm Shift: Fully End-to-End Training for Temporal Sentence Grounding in Videos](https://mlsanity.com/paper/2604.02860/) (2026)

## Abstract (arXiv)

We address the problem of retrieving a specific moment from an untrimmed video by natural language. It is a challenging problem because a target moment may take place in the context of other temporal moments in the untrimmed video. Existing methods cannot tackle this challenge well since they do not fully consider the temporal contexts between temporal moments. In this paper, we model the temporal context between video moments by a set of predefined two-dimensional maps under different temporal scales. For each map, one dimension indicates the starting time of a moment and the other indicates the duration. These 2D temporal maps can cover diverse video moments with different lengths, while representing their adjacent contexts at different temporal scales. Based on the 2D temporal maps, we propose a Multi-Scale Temporal Adjacent Network (MS-2D-TAN), a single-shot framework for moment localization. It is capable of encoding the adjacent temporal contexts at each scale, while learning discriminative features for matching video moments with referring expressions. We evaluate the proposed MS-2D-TAN on three challenging benchmarks, i.e., Charades-STA, ActivityNet Captions, and TACoS, where our MS-2D-TAN outperforms the state of the art.