# Evaluating VLMs' Spatial Reasoning Over Robot Motion: A Step Towards Robot Planning with Motion Preferences

arXiv: https://arxiv.org/abs/2603.13100 · HTML: https://mlsanity.com/paper/2603.13100/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 22 of its 39 references that are in the map.
- First posted on arXiv 13 Mar 2026.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 22 of 22)

- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations](https://mlsanity.com/paper/2301.06052/) (2023)
- [LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action](https://mlsanity.com/paper/2207.04429/) (2022)
- [Open-vocabulary Queryable Scene Representations for Real World Planning](https://mlsanity.com/paper/2209.09874/) (2022)
- [F-VLM: Open-Vocabulary Object Detection upon Frozen Vision and Language Models](https://mlsanity.com/paper/2209.15639/) (2022)
- [Task and Motion Planning with Large Language Models for Object Rearrangement](https://mlsanity.com/paper/2303.06247/) (2023)
- [Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners](https://mlsanity.com/paper/2205.10747/) (2022)
- [Guiding Long-Horizon Task and Motion Planning with Vision Language Models](https://mlsanity.com/paper/2410.02193/) (2024)
- [DocVLM: Make Your VLM an Efficient Reader](https://mlsanity.com/paper/2412.08746/) (2024)
- [LLM-Driven Robots Risk Enacting Discrimination, Violence, and Unlawful Actions](https://mlsanity.com/paper/2406.08824/) (2024)
- [TaskCLIP: Extend Large Vision-Language Model for Task Oriented Object Detection](https://mlsanity.com/paper/2403.08108/) (2024)
- [I Can Tell What I am Doing: Toward Real-World Natural Language Grounding of Robot Experiences](https://mlsanity.com/paper/2411.12960/) (2024)
- [Leveraging Large (Visual) Language Models for Robot 3D Scene Understanding](https://mlsanity.com/paper/2209.05629/) (2022)
- [VLMPlanner: Integrating Visual Language Models with Motion Planning](https://mlsanity.com/paper/2507.20342/) (2025)
- [IMPACT: Intelligent Motion Planning with Acceptable Contact Trajectories via Vision-Language Models](https://mlsanity.com/paper/2503.10110/) (2025)
- [GRAPPA: Generalizing and Adapting Robot Policies via Online Agentic Guidance](https://mlsanity.com/paper/2410.06473/) (2024)

## Abstract (arXiv)

Understanding user instructions and object spatial relations in surrounding environments is crucial for intelligent robot systems to assist humans in various tasks. The natural language and spatial reasoning capabilities of Vision-Language Models (VLMs) have the potential to enhance the generalization of robot planners on new tasks, objects, and motion specifications. While foundation models have been applied to task planning, it is still unclear the degree to which they have the capability of spatial reasoning required to enforce user preferences or constraints on motion, such as desired distances from objects, topological properties, or motion style preferences. In this paper, we evaluate the capability of four state-of-the-art VLMs at spatial reasoning over robot motion, using four different querying methods. Our results show that, with the highest-performing querying method, Qwen2.5-VL achieves 71.4% accuracy zero-shot and 75% on a smaller model after fine-tuning, and GPT-4o leads to lower performance. We evaluate two types of motion preferences (object-proximity and path-style), and we also analyze the trade-off between accuracy and computation cost in number of tokens. This work shows some promise in the potential of VLM integration with robot motion planning pipelines.