mlsanityData to 14 Oct 2026

Evaluating VLMs' Spatial Reasoning Over Robot Motion: A Step Towards Robot Planning with Motion Preferences

arXiv 2603.13100 ↗First posted 13 Mar 2026cs.RO cs.AIcited by 0

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

Understanding user instructions and object spatial relations in surrounding environments is crucial for intelligent robot systems to assist humans in various tasks. The natural language and spatial reasoning capabilities of Vision-Language Models (VLMs) have the potential to enhance the generalization of robot planners on new tasks, objects, and motion specifications. While foundation models have been applied to task planning, it is still unclear the degree to which they have the capability of spatial reasoning required to enforce user preferences or constraints on motion, such as desired distances from objects, topological properties, or motion style preferences. In this paper, we evaluate the capability of four state-of-the-art VLMs at spatial reasoning over robot motion, using four different querying methods. Our results show that, with the highest-performing querying method, Qwen2.5-VL achieves 71.4% accuracy zero-shot and 75% on a smaller model after fine-tuning, and GPT-4o leads to lower performance. We evaluate two types of motion preferences (object-proximity and path-style), and we also analyze the trade-off between accuracy and computation cost in number of tokens. This work shows some promise in the potential of VLM integration with robot motion planning pipelines.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 22 of its 39 references that are in the map. First posted on arXiv 13 Mar 2026.

Builds on

22 of 39 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Task and Motion Planning with Large Language Models for Object Rearrangement10 Mar

Other references · 21

  1. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  3. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  4. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  5. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  6. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  7. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  8. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 588
  9. 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 439
  10. 2022Open-vocabulary Queryable Scene Representations for Real World Planningcited by 160
  11. 2022F-VLM: Open-Vocabulary Object Detection upon Frozen Vision and Language Modelscited by 150
  12. 2022Language Models with Image Descriptors are Strong Few-Shot Video-Language Learnerscited by 117
9 more
  1. 2024Guiding Long-Horizon Task and Motion Planning with Vision Language Modelscited by 47
  2. 2024DocVLM: Make Your VLM an Efficient Readercited by 28
  3. 2024LLM-Driven Robots Risk Enacting Discrimination, Violence, and Unlawful Actionscited by 22
  4. 2024TaskCLIP: Extend Large Vision-Language Model for Task Oriented Object Detectioncited by 21
  5. 2024I Can Tell What I am Doing: Toward Real-World Natural Language Grounding of Robot Experiencescited by 11
  6. 2022Leveraging Large (Visual) Language Models for Robot 3D Scene Understandingcited by 10
  7. 2025VLMPlanner: Integrating Visual Language Models with Motion Planningcited by 6
  8. 2025IMPACT: Intelligent Motion Planning with Acceptable Contact Trajectories via Vision-Language Modelscited by 5
  9. 2024GRAPPA: Generalizing and Adapting Robot Policies via Online Agentic Guidancecited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenes
  2. 2024Structured Spatial Reasoning with Open Vocabulary Object Detectors
  3. 2025Using VLM Reasoning to Constrain Task and Motion Planning
  4. 2024VSP: Assessing the dual challenges of perception and reasoning in spatial planning tasks for VLMs
  5. 2025SpatialCoT: Advancing Spatial Reasoning through Coordinate Alignment and Chain-of-Thought for Embodied Task Planning
  6. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models
  7. 2026ESPIRE: A Diagnostic Benchmark for Embodied Spatial Reasoning of Vision-Language Models
  8. 2026CoPlanVLM: Coordinated Path Planning for a Robot Team using Vision Language Models