# Bridging Time and Space: Decoupled Spatio-Temporal Alignment for Video Grounding arXiv: https://arxiv.org/abs/2604.08014 ยท HTML: https://mlsanity.com/paper/2604.08014/ ## Key facts - Cited by 3 papers in the map (data through 14 Oct 2026). - Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. - Builds on 41 of its 65 references that are in the map. - First posted on arXiv 9 Apr 2026. ## Placed in - Multimodal LLMs / vision-language models / [Video understanding with MLLMs](https://mlsanity.com/mllm/video-understanding-with-mllms/) ## Builds on (most cited first, 30 of 41) - [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014) - [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025) - [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025) - [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025) - [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023) - [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024) - [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025) - [Generation and Comprehension of Unambiguous Object Descriptions](https://mlsanity.com/paper/1511.02283/) (2015) - [VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs](https://mlsanity.com/paper/2406.07476/) (2024) - [NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actions](https://mlsanity.com/paper/2105.08276/) (2021) - [TALL: Temporal Activity Localization via Language Query](https://mlsanity.com/paper/1705.02101/) (2017) - [GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild](https://mlsanity.com/paper/1810.11981/) (2018) - [GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models](https://mlsanity.com/paper/2508.06471/) (2025) - [TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding](https://mlsanity.com/paper/2312.02051/) (2023) - [Seed1.5-VL Technical Report](https://mlsanity.com/paper/2505.07062/) (2025) - [Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding](https://mlsanity.com/paper/2311.08046/) (2023) - [Query-Dependent Video Representation for Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2303.13874/) (2023) - [Where Does It Exist: Spatio-Temporal Video Grounding for Multi-Form Sentences](https://mlsanity.com/paper/2001.06891/) (2020) - [TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuning](https://mlsanity.com/paper/2410.19702/) (2024) - [TRACE: Temporal Grounding Video LLM via Causal Event Modeling](https://mlsanity.com/paper/2410.05643/) (2024) - [HawkEye: Training Video-Text LLMs for Grounding Text in Videos](https://mlsanity.com/paper/2403.10228/) (2024) - [Human-centric Spatio-Temporal Video Grounding With Visual Transformers](https://mlsanity.com/paper/2011.05049/) (2020) - [Knowing Where to Focus: Event-aware Transformer for Video Grounding](https://mlsanity.com/paper/2308.06947/) (2023) - [Autoregressive Queries for Adaptive Tracking with Spatio-TemporalTransformers](https://mlsanity.com/paper/2403.10574/) (2024) - [General Object Foundation Model for Images and Videos at Scale](https://mlsanity.com/paper/2312.09158/) (2023) - [LLaVA-ST: A Multimodal Large Language Model for Fine-Grained Spatial-Temporal Understanding](https://mlsanity.com/paper/2501.08282/) (2025) - [Elysium: Exploring Object-level Perception in Videos via MLLM](https://mlsanity.com/paper/2403.16558/) (2024) - [End-to-end Multi-modal Video Temporal Grounding](https://mlsanity.com/paper/2107.05624/) (2021) - [Context-Guided Spatio-Temporal Video Grounding](https://mlsanity.com/paper/2401.01578/) (2024) - [Embracing Consistency: A One-Stage Approach for Spatio-Temporal Video Grounding](https://mlsanity.com/paper/2209.13306/) (2022) ## Built on by (most cited first, 3 of 3) - [CaC: Advancing Video Reward Models via Hierarchical Spatiotemporal Concentrating](https://mlsanity.com/paper/2605.11723/) (2026) - [Locate Anything in Videos: Rethinking Efficient Generative Spatio-Temporal Video Grounding](https://mlsanity.com/paper/2608.28192/) (2026) - [Pocket-STVG: lightweight architecture for Spatio-Temporal Video Grounding](https://mlsanity.com/paper/2609.31135/) (2026) ## Abstract (arXiv) Spatio-Temporal Video Grounding requires jointly localizing target objects across both temporal and spatial dimensions based on natural language queries, posing fundamental challenges for existing Multimodal Large Language Models (MLLMs). We identify two core challenges: textitentangled spatio-temporal alignment, arising from coupling two heterogeneous sub-tasks within the same autoregressive output space, and textitdual-domain visual token redundancy, where target objects exhibit simultaneous temporal and spatial sparsity, rendering the overwhelming majority of visual tokens irrelevant to the grounding query. To address these, we propose Bridge-STG, an end-to-end framework that decouples temporal and spatial localization while maintaining semantic coherence. While decoupling is the natural solution to this entanglement, it risks creating a semantic gap between the temporal MLLM and the spatial decoder. Bridge-STG resolves this through two pivotal designs: the Spatio-Temporal Semantic Bridging (STSB) mechanism with Explicit Temporal Alignment (ETA) distills the MLLM's temporal reasoning context into enriched bridging queries as a robust semantic interface; and the Query-Guided Spatial Localization (QGSL) module leverages these queries to drive a purpose-built spatial decoder with multi-layer interactive queries and positive/negative frame sampling, jointly eliminating dual-domain visual token redundancy. Extensive experiments across multiple benchmarks demonstrate that Bridge-STG achieves state-of-the-art performance among MLLM-based methods. Bridge-STG improves average mvIoU from 26.4 to 34.3 on VidSTG and demonstrates strong cross-task transfer across various fine-grained video understanding tasks under a unified multi-task training regime.