mlsanityData to 14 Oct 2026

Locate Anything in Videos: Rethinking Efficient Generative Spatio-Temporal Video Grounding

arXiv 2608.28192 ↗First posted 28 Aug 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Spatio-temporal video grounding (STVG) requires models to identify when a referred event occurs and localize the target entity throughout that interval. Existing multimodal large language models typically serialize dense localization trajectories autoregressively, causing decoding latency to grow with tube length and allowing localization errors to propagate across time. We introduce Parallel Tube Decoding (PTD), a generative formulation that decomposes grounding into a temporal block followed by time-conditioned spatial blocks decoded simultaneously. This removes both token-level and trajectory-level dependencies, reducing the sequential decoding depth to a fixed 1 + 1 rounds, independent of tube length. To enable parallel spatial generation, we introduce Decoupled Block Attention, which preserves access to shared video-query context while eliminating cross-box dependencies, together with localization-aware policy optimization for temporal boundaries and spatial geometry. On VidSTG, PTD reduces Tube Completion Latency by 79x and increases spatial decoding throughput by 92x over standard autoregressive decoding, while also improving grounding accuracy. With a compact 4B backbone, our model performs favorably well on VidSTG and HC-STVG, and generalizes zero-shot to temporal grounding, grounded VideoQA, and referring video object tracking. Our results show parallel tube generation is an efficient and effective alternative to autoregressive localization in videos.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 65 of its 66 references that are in the map. First posted on arXiv 28 Aug 2026.

Builds on

65 of 66 references.

References · 65

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2025Qwen2.5-VL Technical Reportcited by 5,587
  4. 2024GPT-4o System Cardcited by 4,588
  5. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  6. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  7. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  8. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  9. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  10. 2025Qwen3-VL Technical Reportcited by 2,793
  11. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,368
  12. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
38 more
  1. 2025Large Language Diffusion Modelscited by 912
  2. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  3. 2025SAM 3: Segment Anything with Conceptscited by 906
  4. 2017Dense-Captioning Events in Videoscited by 793
  5. 2024PaliGemma: A versatile 3B VLM for transfercited by 696
  6. 2017TALL: Temporal Activity Localization via Language Querycited by 584
  7. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
  8. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 409
  9. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 406
  10. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 360
  11. 2021Pix2seq: A Language Modeling Framework for Object Detectioncited by 349
  12. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 332
  13. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 329
  14. 2023VTimeLLM: Empower LLM to Grasp Video Momentscited by 255
  15. 2018Video Object Segmentation with Language Referring Expressionscited by 162
  16. 2025Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videoscited by 159
  17. 2026Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Groundingcited by 139
  18. 2024Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoningcited by 122
  19. 2024Groma: Localized Visual Tokenization for Grounding Multimodal Large Language Modelscited by 119
  20. 2020Where Does It Exist: Spatio-Temporal Video Grounding for Multi-Form Sentencescited by 118
  21. 2024TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuningcited by 106
  22. 2024GroundingGPT:Language Enhanced Multi-modal Grounding Modelcited by 101
  23. 2024One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videoscited by 100
  24. 2022TubeDETR: Spatio-Temporal Video Grounding with Transformerscited by 98
  25. 2024Grounded-VideoLLM: Sharpening Fine-grained Temporal Grounding in Video Large Language Modelscited by 91
  26. 2024TRACE: Temporal Grounding Video LLM via Causal Event Modelingcited by 89
  27. 2020Human-centric Spatio-Temporal Video Grounding With Visual Transformerscited by 85
  28. 2024VTG-LLM: Integrating Timestamp Knowledge into Video LLMs for Enhanced Video Temporal Groundingcited by 85
  29. 2024CG-Bench: Clue-grounded Question Answering Benchmark for Long Video Understandingcited by 85
  30. 2025InternVLA-M1: A Spatially Guided Vision-Language-Action Framework for Generalist Robot Policycited by 84
  31. 2025Detect Anything via Next Point Predictioncited by 73
  32. 2025VideoChat-R1.5: Visual Test-Time Scaling to Reinforce Multimodal Reasoning by Iterative Perceptioncited by 65
  33. 2025LLaVA-ST: A Multimodal Large Language Model for Fine-Grained Spatial-Temporal Understandingcited by 59
  34. 2024ReXTime: A Benchmark Suite for Reasoning-Across-Time in Videoscited by 51
  35. 2024VideoGLaMM: A Large Multimodal Model for Pixel-Level Visual Grounding in Videoscited by 51
  36. 2024Context-Guided Spatio-Temporal Video Groundingcited by 46
  37. 2025Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speedcited by 33
  38. 2024Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignmentcited by 29

50 most cited of 65 All 65 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Pocket-STVG: lightweight architecture for Spatio-Temporal Video Grounding
  2. 2025OmniGround: A Comprehensive Spatio-Temporal Grounding Benchmark for Real-World Complex Scenarios
  3. 2021End-to-End Dense Video Grounding via Parallel Regression
  4. 2025STPro: Spatial and Temporal Progressive Learning for Weakly Supervised Spatio-Temporal Grounding
  5. 2025TubeRMC: Tube-conditioned Reconstruction with Mutual Constraints for Weakly-supervised Spatio-Temporal Video Grounding
  6. 2025Unleashing the Potential of Multimodal LLMs for Zero-Shot Spatio-Temporal Video Grounding
  7. 2026GroundVTS: Visual Token Sampling in Multimodal Large Language Models for Video Temporal Grounding
  8. 2026Agentic Spatio-Temporal Grounding via Collaborative Reasoning