mlsanityData to 14 Oct 2026

STRAND: Benchmarking and Improving Object-Centric Spatio-Temporal Monitoring in Video Large Language Models

arXiv 2609.29607 ↗First posted 28 Aug 2026cs.CV cs.CLcited by 0

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

While multimodal large language models (MLLMs) have advanced video understanding, they remain highly prone to hallucinations in dynamic scenes. We argue this stems from a failure in spatio-temporal monitoring, the ability to persistently track object identities, states, and relations over time. Existing benchmarks obscure this deficit by relying on single final-answer evaluations for queries that can often be resolved via local visual cues or statistical priors. To rigorously diagnose this, we introduce STRAND, a benchmark of human-verified object-centric facts that evaluates intermediate reasoning by decomposing queries into sub-questions, distinguishing genuine temporal understanding from coincidental correctness. Crucially, we score models with Faithful Accuracy, an unconditional joint metric that credits a prediction only when the target answer and every prerequisite sub-question are correct, so that a model cannot inflate its score by being selectively consistent on the small subset of targets it happens to answer correctly. To address failure modes exposed by STRAND, we further propose an object-centric framework that explicitly constructs and reasons over structured object trajectories via chunk-wise state extraction and temporal aggregation. Extensive experiments, including backbone-, frame-, call-, and token-matched comparisons against both end-to-end MLLMs and modular video harnesses, demonstrate that our object-centric framework significantly reduces hallucinated answers and improves spatio-temporal reasoning consistency over state-of-the-art MLLMs. The code, model, and data have been made available at nguyentthong.github.io/strand.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 35 of its 40 references that are in the map. First posted on arXiv 28 Aug 2026.

Builds on

35 of 40 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Video-R1: Reinforcing Video Reasoning in MLLMs27 Mar
  2. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modeling21 Jan
  3. 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modeling31 Dec
  4. 2024Apollo: An Exploration of Video Understanding in Large Multimodal Models13 Dec
  5. 2024LVBench: An Extreme Long Video Understanding Benchmark12 Jun
  6. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis31 May
  7. 2024TraveLER: A Modular Multi-LMM Agent Framework for Video Question-Answering1 Apr

Other references · 28

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2025Qwen3-VL Technical Reportcited by 2,793
  3. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  4. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  5. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
  6. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 290
  7. 2026Gemma 4 Technical Reportcited by 259
  8. 2023Video Understanding with Large Language Models: A Surveycited by 206
  9. 2023Self-Chained Image-Language Model for Video Localization and Question Answeringcited by 203
  10. 2026Qwen3.5-Omni Technical Reportcited by 174
  11. 2025Cosmos-Reason1: From Physical Common Sense To Embodied Reasoningcited by 164
  12. 2025VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuningcited by 93
16 more
  1. 2024VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Modelscited by 90
  2. 2025STI-Bench: Are MLLMs Ready for Precise Spatial-Temporal World Understanding?cited by 79
  3. 2024VideoRefer Suite: Advancing Spatial-Temporal Object Understanding with Video LLMcited by 72
  4. 2024VidHalluc: Evaluating Temporal Hallucinations in Multimodal Large Language Models for Video Understandingcited by 68
  5. 2025V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoningcited by 68
  6. 2024Grounded Multi-Hop VideoQA in Long-Form Egocentric Videoscited by 45
  7. 2025VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded Video Reasoningcited by 43
  8. 2026Dive into Claude Code: The Design Space of Today's and Future AI Agent Systemscited by 38
  9. 2025VideoChat-A1: Thinking with Long Videos by Chain-of-Shot Reasoningcited by 32
  10. 2025MASH-VLM: Mitigating Action-Scene Hallucination in Video-LLMs through Disentangled Spatial-Temporal Representationscited by 22
  11. 2025Exploring Hallucination of Large Multimodal Models in Video Understanding: Benchmark, Analysis and Mitigationcited by 20
  12. 2025ARGUS: Hallucination and Omission Evaluation in Video-LLMscited by 16
  13. 2025Object-centric Video Question Answering with Visual Grounding and Referringcited by 13
  14. 2026TimeBlind: A Spatio-Temporal Compositionality Benchmark for Video LLMscited by 11
  15. 2025ELV-Halluc: Benchmarking Semantic Aggregation Hallucinations in Long Video Understandingcited by 10
  16. 2025SmartSight: Mitigating Hallucination in Video-LLMs Without Compromising Video Understanding via Temporal Attention Collapsecited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Tracking the Truth: Object-Centric Spatio-Temporal Monitoring for Video Large Language Models
  2. 2026TOC-Bench: A Temporal Object Consistency Benchmark for Video Large Language Models
  3. 2026Video-MME-Logical: A Controlled Diagnostic Benchmark for Video Temporal-Logical Reasoning
  4. 2025Know-Show: Benchmarking Video-Language Models on Spatio-Temporal Grounded Reasoning
  5. 2026Moment-Video: Diagnosing Temporal Fidelity of Video MLLMs on Momentary Visual Events
  6. 2026CaST-Bench: Benchmarking Causal Chain-Grounded Spatio-Temporal Reasoning for Video Question Answering
  7. 2026VideoZeroBench: Probing the Limits of Video MLLMs with Spatio-Temporal Evidence Verification
  8. 2026SYNCR: A Cross-Video Reasoning Benchmark with Synthetic Grounding