mlsanityData to 7 Oct 2026

TALL: Temporal Activity Localization via Language Query

arXiv 1705.02101 ↗First posted 5 May 2017cs.CVcited by 581

Not on the map. Why?

Cited by 90 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This paper focuses on temporal localization of actions in untrimmed videos. Existing methods typically train classifiers for a pre-defined list of actions and apply them in a sliding window fashion. However, activities in the wild consist of a wide combination of actors, actions and objects; it is difficult to design a proper activity list that meets users' needs. We propose to localize activities by natural language queries. Temporal Activity Localization via Language (TALL) is challenging as it requires: (1) suitable design of text and video representations to allow cross-modal matching of actions and language queries; (2) ability to locate actions accurately given features from sliding windows of limited granularity. We propose a novel Cross-modal Temporal Regression Localizer (CTRL) to jointly model text query and video clips, output alignment scores and action boundary regression results for candidate clips. For evaluation, we adopt TaCoS dataset, and build a new dataset for this task on top of Charades by adding sentence temporal annotations, called Charades-STA. We also build complex sentence queries in Charades-STA for test. Experimental results show that CTRL outperforms previous methods significantly on both datasets.

Key facts. Cited by 581 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 57; 2022: 68; 2023: 91; 2024: 94; 2025: 147; 2026: 124.

581 papers.

Sort:

By area:

  1. 2025Qwen3-VL Technical Reportcited by 2,534
  2. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  3. 2023VideoChat: Chat-Centric Video Understandingcited by 851
  4. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
  5. 2021Less is More: ClipBERT for Video-and-Language Learning via Sparse Samplingcited by 484
  6. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 399
  7. 2024VideoMamba: State Space Model for Efficient Video Understandingcited by 325
  8. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 310
  9. 2023VTimeLLM: Empower LLM to Grasp Video Momentscited by 253
  10. 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuningcited by 248
  11. 2022Egocentric Video-Language Pretrainingcited by 231
  12. 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modelingcited by 212
38 more
  1. 2023Video Understanding with Large Language Models: A Surveycited by 203
  2. 2023TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesiscited by 194
  3. 2023UniVTG: Towards Unified Video-Language Temporal Groundingcited by 189
  4. 2021VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modelingcited by 187
  5. 2023Query-Dependent Video Representation for Moment Retrieval and Highlight Detectioncited by 166
  6. 2025SpaceR: Reinforcing MLLMs in Video Spatial Reasoningcited by 152
  7. 2022UMT: Unified Multi-modal Transformers for Joint Video Moment Retrieval and Highlight Detectioncited by 148
  8. 2025Time-R1: Post-Training Large Vision Language Model for Temporal Video Groundingcited by 138
  9. 2023Can I Trust Your Answer? Visually Grounded Video Question Answeringcited by 133
  10. 2021A Survey on Deep Learning Technique for Video Segmentationcited by 132
  11. 2023EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbonecited by 130
  12. 2024Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoningcited by 122
  13. 2025VLM2Vec-V2: Advancing Multimodal Embedding for Videos, Images, and Visual Documentscited by 110
  14. 2021Context-aware Biaffine Localizing Network for Temporal Sentence Groundingcited by 107
  15. 2021MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptionscited by 107
  16. 2024Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understandingcited by 106
  17. 2025MiMo-VL Technical Reportcited by 106
  18. 2024VideoPrism: A Foundational Visual Encoder for Video Understandingcited by 103
  19. 2022Animal Kingdom: A Large and Diverse Dataset for Animal Behavior Understandingcited by 102
  20. 2024GroundingGPT:Language Enhanced Multi-modal Grounding Modelcited by 101
  21. 2025Daily-Omni: Towards Audio-Visual Reasoning with Temporal Alignment across Modalitiescited by 101
  22. 2024TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuningcited by 99
  23. 2022TubeDETR: Spatio-Temporal Video Grounding with Transformerscited by 97
  24. 2024TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Modelscited by 96
  25. 2021VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluationcited by 95
  26. 2021Interventional Video Grounding with Dual Contrastive Learningcited by 94
  27. 2021Negative Sample Matters: A Renaissance of Metric Learning for Temporal Groundingcited by 90
  28. 2021Boundary Proposal Network for Two-Stage Natural Language Video Localizationcited by 88
  29. 2025Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoningcited by 88
  30. 2024Grounded-VideoLLM: Sharpening Fine-grained Temporal Grounding in Video Large Language Modelscited by 87
  31. 2024TRACE: Temporal Grounding Video LLM via Causal Event Modelingcited by 87
  32. 2024MileBench: Benchmarking MLLMs in Long Contextcited by 86
  33. 2021Deconfounded Video Moment Retrieval with Causal Interventioncited by 84
  34. 2024HawkEye: Training Video-Text LLMs for Grounding Text in Videoscited by 83
  35. 2023Knowing Where to Focus: Event-aware Transformer for Video Groundingcited by 79
  36. 2022An Empirical Study of End-to-End Video-Language Transformers with Masked Visual Modelingcited by 74
  37. 2024A Survey of Multimodal Large Language Model from A Data-centric Perspectivecited by 73
  38. 2021QVHighlights: Detecting Moments and Highlights in Videos via Natural Language Queriescited by 72

50 most cited of 581 All 581 (JSON)