mlsanityData to 7 Oct 2026

R3D: Quantitative 3D Spatial Reasoning for Egocentric Wearables

arXiv 2607.02921 ↗First posted 3 Jul 2026cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Quantitative 3D spatial reasoning from egocentric RGB-D video is a critical capability for next-generation wearable assistants. Yet existing benchmarks do not reflect the challenges of handling (1) natural egocentric video, (2) posed RGB-D video inputs, and (3) challenging quantitative 3D spatial reasoning Q&A. To fill this gap, we introduce R3D-Bench (Reasoning in 3D), a benchmark of 3,033 quantitative spatial reasoning questions across 15 types -- spanning multiple-choice, distance-based, and volumetric reasoning questions -- built on top of 57 egocentric video sequences from Aria Digital Twin. To set a strong baseline on this dataset, we introduce R3D, a model-agnostic spatial tool-calling framework. In contrast to existing approaches that directly embed 3D information into the model's input representation, R3D constructs a 3D scene from video using segmentation and depth-lifted object representations. It provides this information to an LLM through eight composable spatial tools. On R3D-Bench, R3D with Qwen3-VL 235B achieves 73.5% mean relative accuracy, substantially outperforming the best depth-enabled baseline (CuTR+Tools, 61.9%) and the best RGB-only baseline (Gemini 3 Flash, 46.5%).

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 31 of its 37 references that are in the map. First posted on arXiv 3 Jul 2026.

Builds on

31 of 37 references.

References · 31

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  5. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  6. 2025SAM 3: Segment Anything with Conceptscited by 835
  7. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 747
  8. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
  9. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
  10. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 556
  11. 2023LERF: Language Embedded Radiance Fieldscited by 538
  12. 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 357
19 more
  1. 2023An Embodied Generalist Agent in 3D Worldcited by 355
  2. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 340
  3. 2022SQA3D: Situated Question Answering in 3D Scenescited by 303
  4. 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 302
  5. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Modelscited by 297
  6. 2025SAM 3D: 3Dfy Anything in Imagescited by 230
  7. 2023Project Aria: A New Tool for Egocentric Multi-Modal AI Researchcited by 201
  8. 2023Aria Digital Twin: A New Benchmark Dataset for Egocentric 3D Machine Perceptioncited by 166
  9. 2023EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AIcited by 146
  10. 2024Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understandingcited by 131
  11. 2025MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMscited by 72
  12. 2025Visual Agentic AI for Spatial Reasoning with a Dynamic APIcited by 42
  13. 2026Think3D: Thinking with Space for Spatial Reasoningcited by 22
  14. 2025SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RLcited by 20
  15. 2026SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoningcited by 13
  16. 2026pySpatial: Generating 3D Visual Programs for Zero-Shot Spatial Reasoningcited by 12
  17. 2025From Indoor to Open World: Revealing the Spatial Reasoning Gap in MLLMscited by 10
  18. 2026S-Agent: Spatial Tool-Use Elicits Reasoning for Spatial Intelligencecited by 7
  19. 2026RieMind: Geometry-Grounded Spatial Agent for Scene Understandingcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 20243DSRBench: A Comprehensive 3D Spatial Reasoning Benchmark
  2. 2026Spatial-Conditioned Reasoning in Long-Egocentric Videos
  3. 2025Out of Sight, Not Out of Context? Egocentric Spatial Reasoning in VLMs Across Disjoint Frames
  4. 2025Spatial Reasoning with Vision-Language Models in Ego-Centric Multi-View Scenes
  5. 2026EgoProx: Evaluating MLLMs on Egocentric 3D Proximity Reasoning Across a Cognitive Hierarchy
  6. 2026Long Time No See: Benchmarking VLMs for Out-of-Sight Spatiotemporal Reasoning in Egocentric Videos
  7. 2026Minerva-Ego: Spatiotemporal Hints for Egocentric Video Understanding
  8. 2025SURPRISE3D: A Dataset for Spatial Understanding and Reasoning in Complex 3D Scenes