OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMs
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
Abstract (arXiv)
Multimodal agents in robotics, AR, and autonomous driving must reason about places and layouts from continuous egocentric streams, often using evidence outside the current view. Existing benchmarks either evaluate offline over full videos or target events rather than spatial structure. We introduce OVO-S-Bench, a fully human-annotated benchmark for streaming spatial intelligence, comprising 1,680 questions over 348 source videos. Annotation involves 12 trained annotators (each also serving as a blind cross-reviewer) across roughly 804 person-hours of multi-round quality assurance. Each question carries a query timestamp and an evidence interval, and at evaluation, the model sees only the prefix preceding the query. Questions span four levels of increasing abstraction: instantaneous egocentric perception, spatiotemporal context tracking, generative spatial reasoning, and allocentric spatial mapping. Across 38 proprietary and open-source MLLMs, Gemini-3.1-Pro trails human experts evaluated under the same prefix-access protocol by 33 points (59.2 vs. 92.2), with allocentric spatial mapping as the dominant bottleneck. Notably, streaming and spatially fine-tuned MLLMs underperform their own backbones. We further find that chain-of-thought reasoning amplifies spatial errors when ungrounded in the stream. By exposing these limitations, OVO-S-Bench establishes a demanding testbed for next-generation streaming spatial MLLMs.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 58 of its 65 references that are in the map. First posted on arXiv 2 Jun 2026.
Builds on
58 of 65 references.
Closest ancestors · 15
Same direction, newest first
- 2025SpaceMind: Camera-Guided Modality Fusion for Spatial Reasoning in Vision-Language Models28 Nov
- 2025SpatialBench: Benchmarking Multimodal Large Language Models for Spatial Cognition26 Nov
- 2025Robobench: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models as Embodied Brain20 Oct
- 2025OST-Bench: Evaluating the Capabilities of MLLMs in Online Spatio-temporal Scene Understanding10 Jul
- 2025OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Models3 Jun
- 2025Visual Embodied Brain: Let Multimodal Large Language Models See, Think, and Control in Spaces30 May
- 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence29 May
- 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence29 May
- 2025ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models27 May
- 2025STI-Bench: Are MLLMs Ready for Precise Spatial-Temporal World Understanding?31 Mar
- 2025Open3D-VQA: A Benchmark for Comprehensive Spatial Reasoning with Multimodal Large Language Model in Open Space14 Mar
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec
3 more
Other references · 43
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
- 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
- 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 665
- 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 467
- 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 454
- 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 410
- 2024Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streamscited by 169
- 2024Streaming Long Video Understanding with Large Language Modelscited by 167
- 2025Cosmos-Reason1: From Physical Common Sense To Embodied Reasoningcited by 164
- 2024StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understandingcited by 144
31 more
- 2025OVO-Bench: How Far is Your Video-LLMs from Real-World Online Video Understanding?cited by 142
- 2025Cambrian-S: Towards Spatial Supersensing in Videocited by 134
- 2025Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reactioncited by 127
- 2018Toward Driving Scene Understanding: A Dataset for Learning Driver Behavior and Causal Reasoningcited by 102
- 2022CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Drivingcited by 100
- 2025TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videoscited by 97
- 2025StreamingVLM: Real-Time Understanding for Infinite Video Streamscited by 95
- 2025Visual Spatial Tuningcited by 89
- 2025Scaling Spatial Intelligence with Multimodal Foundation Modelscited by 82
- 2025Flash-VStream: Efficient Real-Time Understanding for Long Video Streamscited by 81
- 2025StreamForest: Efficient Online Video Understanding with Persistent Event Memorycited by 78
- 2024Online Video Understanding: OVBench and VideoChat-Onlinecited by 61
- 2025InfiniPot-V: Memory-Constrained KV Cache Compression for Streaming Video Understandingcited by 58
- 2026Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understandingcited by 55
- 2025StreamingTOM: Streaming Token Compression for Efficient Video Understandingcited by 43
- 2026HERMES: KV Cache as Hierarchical Memory for Efficient Streaming Video Understandingcited by 41
- 2024Aria Everyday Activities Datasetcited by 39
- 2026RoboBrain 2.5: Depth in Sight, Time in Mindcited by 39
- 2025MMSI-Video-Bench: A Holistic Benchmark for Video-Based Spatial Intelligencecited by 38
- 2026RynnBrain: Open Embodied Foundation Modelscited by 35
- 2026A Simple Baseline for Streaming Video Understandingcited by 33
- 2026FluxMem: Adaptive Hierarchical Memory for Streaming Video Understandingcited by 28
- 2026ESI-Bench: Towards Embodied Spatial Intelligence that Closes the Perception-Action Loopcited by 26
- 2024RoomTour3D: Geometry-Aware Video-Instruction Tuning for Embodied Navigationcited by 23
- 2025DSI-Bench: A Benchmark for Dynamic Spatial Intelligencecited by 23
- 2025StreamEQA: Towards Streaming Video Understanding for Embodied Scenarioscited by 12
- 2025SpatialTree: How Spatial Abilities Branch Out in MLLMscited by 12
- 2026Spatial-TTT: Streaming Visual-based Spatial Intelligence with Test-Time Trainingcited by 11
- 2025Out of Sight, Not Out of Context? Egocentric Spatial Reasoning in VLMs Across Disjoint Framescited by 6
- 2026Holi-Spatial: Evolving Video Streams into Holistic 3D Spatial Intelligencecited by 6
- 2026SVCBench: A Streaming Video Counting Benchmark for Spatial-Temporal State Maintenancecited by 6
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026TRACE: Temporal Retrieval with Anchored and Convergent Evidence for Long-Horizon Video Understandingcited by 2
- 2026STORM-Bench: Evaluating Online Video QA under Evolving and Incomplete Evidencecited by 0
- 2026When Should the Count Change? Learning State Maintenance for Causal Video Countingcited by 0
- 2026Revision, Not Restart: Revisable Visual Plans for Closed-Loop World-Action Modelscited by 0
- 2026HumanPhys: A Human-Centric Physical-World Perception Benchmark for Real-World Visual Reasoningcited by 0
- 2026ReTAC: Residual Test-Time Agentic Computation for Video Spatial Reasoningcited by 0
- 2026ReTAC: Residual Test-Time Agentic Computation for Video Spatial Reasoning12 Oct 2026
- 2026HumanPhys: A Human-Centric Physical-World Perception Benchmark for Real-World Visual Reasoning10 Oct 2026
- 2026Revision, Not Restart: Revisable Visual Plans for Closed-Loop World-Action Models28 Sep 2026
- 2026When Should the Count Change? Learning State Maintenance for Causal Video Counting28 Sep 2026
- 2026STORM-Bench: Evaluating Online Video QA under Evolving and Incomplete Evidence25 Sep 2026
- 2026TRACE: Temporal Retrieval with Anchored and Convergent Evidence for Long-Horizon Video Understanding23 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2026See, Remember, Explore: A Benchmark and Baselines for Streaming Spatial Reasoning
- 2026Keep It in Mind: User Centric Continual Spatial Intelligence Reasoning in Egocentric Video Streams
- 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks
- 2026DRIVESPATIAL: A Benchmark for Spatiotemporal Intelligence in VLMs for Autonomous Driving
- 2025SpatialScore: Towards Comprehensive Evaluation for Spatial Intelligence
- 2026SpaceCast-Bench: Evaluating Predictive Spatial Reasoning in Vision-Language Models
- 2026From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs
- 2026Minerva-Ego: Spatiotemporal Hints for Egocentric Video Understanding