# Tracking the Truth: Object-Centric Spatio-Temporal Monitoring for Video Large Language Models

arXiv: https://arxiv.org/abs/2605.08974 · HTML: https://mlsanity.com/paper/2605.08974/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 30 of its 36 references that are in the map.
- First posted on arXiv 9 May 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Video understanding with MLLMs](https://mlsanity.com/mllm/video-understanding-with-mllms/)

## Builds on (most cited first, 30 of 30)

- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [A Survey on Multimodal Large Language Models](https://mlsanity.com/paper/2306.13549/) (2023)
- [EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding](https://mlsanity.com/paper/2308.09126/) (2023)
- [LLaVA-Video: Video Instruction Tuning With Synthetic Data](https://mlsanity.com/paper/2410.02713/) (2024)
- [Video-R1: Reinforcing Video Reasoning in MLLMs](https://mlsanity.com/paper/2503.21776/) (2025)
- [VideoChat-Flash: Hierarchical Compression for Long-Context Video Modeling](https://mlsanity.com/paper/2501.00574/) (2024)
- [Video Understanding with Large Language Models: A Survey](https://mlsanity.com/paper/2312.17432/) (2023)
- [Self-Chained Image-Language Model for Video Localization and Question Answering](https://mlsanity.com/paper/2305.06988/) (2023)
- [Qwen3.5-Omni Technical Report](https://mlsanity.com/paper/2604.15804/) (2026)
- [VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuning](https://mlsanity.com/paper/2505.12434/) (2025)
- [VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Models](https://mlsanity.com/paper/2406.16338/) (2024)
- [Apollo: An Exploration of Video Understanding in Large Multimodal Models](https://mlsanity.com/paper/2412.10360/) (2024)
- [STI-Bench: Are MLLMs Ready for Precise Spatial-Temporal World Understanding?](https://mlsanity.com/paper/2503.23765/) (2025)
- [VideoRefer Suite: Advancing Spatial-Temporal Object Understanding with Video LLM](https://mlsanity.com/paper/2501.00599/) (2024)
- [VidHalluc: Evaluating Temporal Hallucinations in Multimodal Large Language Models for Video Understanding](https://mlsanity.com/paper/2412.03735/) (2024)
- [V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoning](https://mlsanity.com/paper/2503.11495/) (2025)
- [VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded Video Reasoning](https://mlsanity.com/paper/2503.13444/) (2025)
- [VideoChat-A1: Thinking with Long Videos by Chain-of-Shot Reasoning](https://mlsanity.com/paper/2506.06097/) (2025)
- [Understanding Long Videos with Multimodal Language Models](https://mlsanity.com/paper/2403.16998/) (2024)
- [TraveLER: A Modular Multi-LMM Agent Framework for Video Question-Answering](https://mlsanity.com/paper/2404.01476/) (2024)
- [MASH-VLM: Mitigating Action-Scene Hallucination in Video-LLMs through Disentangled Spatial-Temporal Representations](https://mlsanity.com/paper/2503.15871/) (2025)
- [Exploring Hallucination of Large Multimodal Models in Video Understanding: Benchmark, Analysis and Mitigation](https://mlsanity.com/paper/2503.19622/) (2025)
- [ARGUS: Hallucination and Omission Evaluation in Video-LLMs](https://mlsanity.com/paper/2506.07371/) (2025)
- [TimeBlind: A Spatio-Temporal Compositionality Benchmark for Video LLMs](https://mlsanity.com/paper/2602.00288/) (2026)
- [ELV-Halluc: Benchmarking Semantic Aggregation Hallucinations in Long Video Understanding](https://mlsanity.com/paper/2508.21496/) (2025)
- [SmartSight: Mitigating Hallucination in Video-LLMs Without Compromising Video Understanding via Temporal Attention Collapse](https://mlsanity.com/paper/2512.18671/) (2025)
- [Enhancing Video Representations with Spatiotemporal-Semantic Residual to Mitigate Hallucinations in Video Large Multimodal Models](https://mlsanity.com/paper/2601.22574/) (2026)

## Abstract (arXiv)

While multimodal large language models (MLLMs) have advanced video understanding, they remain highly prone to hallucinations in dynamic scenes. We argue this stems from a failure in spatio-temporal monitoring, the ability to persistently track object identities, states, and relations over time. Existing benchmarks obscure this deficit by relying on single final-answer evaluations for queries that can often be resolved via local visual cues or statistical priors. To rigorously diagnose this, we introduce STEMO-Bench (Spatio-TEmporal MOnitoring), a benchmark of human-verified object-centric facts that evaluates intermediate reasoning by decomposing queries into sub-questions, distinguishing genuine temporal understanding from coincidental correctness. To address failure modes exposed by STEMO, we propose STEMO-Track, a novel object-centric framework that explicitly constructs and reasons over structured object trajectories via chunk-wise state extraction and temporal aggregation. Extensive experiments demonstrate that our object-centric framework significantly reduces hallucinated answers and improves spatio-temporal reasoning consistency over state-of-the-art MLLMs.