R3D: Quantitative 3D Spatial Reasoning for Egocentric Wearables
Not on the map. Why?
Abstract (arXiv)
Quantitative 3D spatial reasoning from egocentric RGB-D video is a critical capability for next-generation wearable assistants. Yet existing benchmarks do not reflect the challenges of handling (1) natural egocentric video, (2) posed RGB-D video inputs, and (3) challenging quantitative 3D spatial reasoning Q&A. To fill this gap, we introduce R3D-Bench (Reasoning in 3D), a benchmark of 3,033 quantitative spatial reasoning questions across 15 types -- spanning multiple-choice, distance-based, and volumetric reasoning questions -- built on top of 57 egocentric video sequences from Aria Digital Twin. To set a strong baseline on this dataset, we introduce R3D, a model-agnostic spatial tool-calling framework. In contrast to existing approaches that directly embed 3D information into the model's input representation, R3D constructs a 3D scene from video using segmentation and depth-lifted object representations. It provides this information to an LLM through eight composable spatial tools. On R3D-Bench, R3D with Qwen3-VL 235B achieves 73.5% mean relative accuracy, substantially outperforming the best depth-enabled baseline (CuTR+Tools, 61.9%) and the best RGB-only baseline (Gemini 3 Flash, 46.5%).
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 31 of its 37 references that are in the map. First posted on arXiv 3 Jul 2026.
Builds on
31 of 37 references.
References · 31
- 2023GPT-4 Technical Reportcited by 18,132
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
- 2025SAM 3: Segment Anything with Conceptscited by 835
- 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 747
- 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
- 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 556
- 2023LERF: Language Embedded Radiance Fieldscited by 538
- 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 357
19 more
- 2023An Embodied Generalist Agent in 3D Worldcited by 355
- 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 340
- 2022SQA3D: Situated Question Answering in 3D Scenescited by 303
- 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 302
- 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Modelscited by 297
- 2025SAM 3D: 3Dfy Anything in Imagescited by 230
- 2023Project Aria: A New Tool for Egocentric Multi-Modal AI Researchcited by 201
- 2023Aria Digital Twin: A New Benchmark Dataset for Egocentric 3D Machine Perceptioncited by 166
- 2023EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AIcited by 146
- 2024Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understandingcited by 131
- 2025MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMscited by 72
- 2025Visual Agentic AI for Spatial Reasoning with a Dynamic APIcited by 42
- 2026Think3D: Thinking with Space for Spatial Reasoningcited by 22
- 2025SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RLcited by 20
- 2026SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoningcited by 13
- 2026pySpatial: Generating 3D Visual Programs for Zero-Shot Spatial Reasoningcited by 12
- 2025From Indoor to Open World: Revealing the Spatial Reasoning Gap in MLLMscited by 10
- 2026S-Agent: Spatial Tool-Use Elicits Reasoning for Spatial Intelligencecited by 7
- 2026RieMind: Geometry-Grounded Spatial Agent for Scene Understandingcited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 20243DSRBench: A Comprehensive 3D Spatial Reasoning Benchmark
- 2026Spatial-Conditioned Reasoning in Long-Egocentric Videos
- 2025Out of Sight, Not Out of Context? Egocentric Spatial Reasoning in VLMs Across Disjoint Frames
- 2025Spatial Reasoning with Vision-Language Models in Ego-Centric Multi-View Scenes
- 2026EgoProx: Evaluating MLLMs on Egocentric 3D Proximity Reasoning Across a Cognitive Hierarchy
- 2026Long Time No See: Benchmarking VLMs for Out-of-Sight Spatiotemporal Reasoning in Egocentric Videos
- 2026Minerva-Ego: Spatiotemporal Hints for Egocentric Video Understanding
- 2025SURPRISE3D: A Dataset for Spatial Understanding and Reasoning in Complex 3D Scenes