mlsanityData to 7 Oct 2026

MosaicThinker: On-Device Visual Spatial Reasoning for Embodied AI via Iterative Construction of Space Representation

arXiv 2602.07082 ↗First posted 6 Feb 2026cs.CV cs.AIcited by 2

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

When embodied AI is expanding from traditional object detection and recognition to more advanced tasks of robot manipulation and actuation planning, visual spatial reasoning from the video inputs is necessary to perceive the spatial relationships of objects and guide device actions. However, existing visual language models (VLMs) have very weak capabilities in spatial reasoning due to the lack of knowledge about 3D spatial information, especially when the reasoning task involve complex spatial relations across multiple video frames. In this paper, we present a new inference-time computing technique for on-device embodied AI, namely MosaicThinker, which enhances the on-device small VLM's spatial reasoning capabilities on difficult cross-frame reasoning tasks. Our basic idea is to integrate fragmented spatial information from multiple frames into a unified space representation of global semantic map, and further guide the VLM's spatial reasoning over the semantic map via a visual prompt. Experiment results show that our technique can greatly enhance the accuracy of cross-frame spatial reasoning on resource-constrained embodied AI devices, over reasoning tasks with diverse types and complexities.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 72 of its 98 references that are in the map. First posted on arXiv 6 Feb 2026.

Builds on

72 of 98 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025InfiniBench: Infinite Benchmarking for Visual Spatial Reasoning with Customizable Scene Complexity22 Nov
  2. 2025SIMS-V: Simulated Instruction-Tuning for Spatial Video Understanding6 Nov
  3. 20253DRS: MLLMs Need 3D-Aware Representation Supervision for Scene Understanding2 Jun
  4. 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence29 May
  5. 2025SpaRE: Enhancing Spatial Reasoning in Vision-Language Models with Synthetic Data29 Apr
  6. 2025Perspective-Aware Reasoning in Vision-Language Models via Mental Imagery Simulation24 Apr
  7. 2025Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Tasks27 Mar
  8. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec
  9. 2024Multi-modal Situated Reasoning in 3D Scenes4 Sep
  10. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan
  11. 2023Exploring and Improving the Spatial Reasoning Abilities of Large Language Models2 Dec
  12. 2023LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning30 Nov

Other references · 60

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 2023Segment Anythingcited by 8,158
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2024GPT-4o System Cardcited by 4,467
  5. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  6. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  7. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  8. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  9. 2024Depth Anything V2cited by 1,385
  10. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
  11. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
  12. 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 498
38 more
  1. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 395
  2. 2017Embodied Question Answeringcited by 335
  3. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 323
  4. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 312
  5. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  6. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 256
  7. 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligencecited by 206
  8. 2024MuirBench: A Comprehensive Benchmark for Robust Multi-image Understandingcited by 178
  9. 2024LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awarenesscited by 178
  10. 2024MobileVLM V2: Faster and Stronger Baseline for Vision Language Modelcited by 172
  11. 2023EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AIcited by 146
  12. 2024Vision language models are blind: Failing to translate detailed visual features into wordscited by 144
  13. 2023LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an Agentcited by 134
  14. 2024Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understandingcited by 131
  15. 2025Cambrian-S: Towards Spatial Supersensing in Videocited by 123
  16. 2021UniTAB: Unifying Text and Box Outputs for Grounded Vision-Language Modelingcited by 122
  17. 2025Learning from Videos for 3D World: Enhancing MLLMs with 3D Vision Geometry Priorscited by 111
  18. 2025GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Modelscited by 106
  19. 2023LiDAR-LLM: Exploring the Potential of Large Language Models for 3D LiDAR Understandingcited by 95
  20. 2025MindCube: Spatial Mental Modeling from Limited Viewscited by 91
  21. 2025T*: Re-thinking Temporal Search for Long-Form Video Understandingcited by 82
  22. 2025STI-Bench: Are MLLMs Ready for Precise Spatial-Temporal World Understanding?cited by 74
  23. 2023Evaluating Spatial Understanding of Large Language Modelscited by 65
  24. 2025Ross3D: Reconstructive Visual Instruction Tuning with 3D-Awarenesscited by 61
  25. 2023PerceptionGPT: Effectively Fusing Visual Perception into LLMcited by 59
  26. 2024Scaffolding Coordinates to Promote Vision-Language Coordination in Large Multi-Modal Modelscited by 56
  27. 2025Inst3D-LMM: Instance-Aware 3D Scene Understanding with Multi-modal Instruction Tuningcited by 50
  28. 2023Panoptic Video Scene Graph Generationcited by 46
  29. 2024Embodied-RAG: General Non-parametric Embodied Memory for Retrieval and Generationcited by 44
  30. 2025Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Modelscited by 44
  31. 2023MobileSAMv2: Faster Segment Anything to Everythingcited by 41
  32. 2025MatchAnything: Universal Cross-Modality Image Matching with Large-Scale Pre-Trainingcited by 40
  33. 2024LSceneLLM: Enhancing Large 3D Scene Understanding Using Adaptive Visual Preferencescited by 36
  34. 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulationcited by 34
  35. 2024Temporally Consistent Object-Centric Learning by Contrasting Slotscited by 30
  36. 2025CoS: Chain-of-Shot Prompting for Long Video Understandingcited by 27
  37. 2024Language-driven Grasp Detectioncited by 26
  38. 2024Cross-Embodiment Dexterous Grasping with Reinforcement Learningcited by 24

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026SpaMEM: Benchmarking Dynamic Spatial Reasoning via Perception-Memory Integration in Embodied Environmentscited by 6
  2. 2026Uncovering and Shaping the Latent Representation of 3D Scene Topology in Vision-Language Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025SpatialCoT: Advancing Spatial Reasoning through Coordinate Alignment and Chain-of-Thought for Embodied Task Planning
  2. 2025EmbodiedVSR: Dynamic Scene Graph-Guided Chain-of-Thought Reasoning for Visual Spatial Tasks
  3. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks
  4. 2025Embodied Spatial Intelligence: from Implicit Scene Modeling to Spatial Reasoning
  5. 2026Chain of Spatial Thoughts: Modality-Agnostic Spatial Grounding for Vision Language Models
  6. 2025REM: Evaluating LLM Embodied Spatial Reasoning through Multi-Frame Trajectories
  7. 2025Towards Embodied Cognition in Robots via Spatially Grounded Synthetic Worlds
  8. 2026ESPIRE: A Diagnostic Benchmark for Embodied Spatial Reasoning of Vision-Language Models