# Using VLM Reasoning to Constrain Task and Motion Planning

arXiv: https://arxiv.org/abs/2510.25548 · HTML: https://mlsanity.com/paper/2510.25548/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 6 of its 37 references that are in the map.
- First posted on arXiv 29 Oct 2025.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 6 of 6)

- [Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planning](https://mlsanity.com/paper/2311.17842/) (2023)
- [AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic Manipulation](https://mlsanity.com/paper/2410.00371/) (2024)
- [Grounding Classical Task Planners via Vision-Language Models](https://mlsanity.com/paper/2304.08587/) (2023)
- [A Survey on Large Language Models for Automated Planning](https://mlsanity.com/paper/2502.12435/) (2025)
- [Accelerating Integrated Task and Motion Planning with Neural Feasibility Checking](https://mlsanity.com/paper/2203.10568/) (2022)
- [Open-World Task and Motion Planning via Vision-Language Model Generated Constraints](https://mlsanity.com/paper/2411.08253/) (2024)

## Built on by (most cited first, 2 of 2)

- [APIVOT: Adaptive Planning with Interleaved Vision-Language Thoughts](https://mlsanity.com/paper/2607.08024/) (2026)
- [PDDL-ART: Autonomous Symbolic Abstraction From Demonstration For Long-Horizon Robotic Manipulation Using Vision-Language Models](https://mlsanity.com/paper/2608.17146/) (2026)

## Abstract (arXiv)

In task and motion planning, high-level task planning is done over an abstraction of the world to enable efficient search in long-horizon robotics problems. However, the feasibility of these task-level plans relies on the downward refinability of the abstraction into continuous motion. When a domain's refinability is poor, task-level plans that appear valid may ultimately fail during motion planning, requiring replanning and resulting in slower overall performance. Prior works mitigate this by encoding refinement issues as constraints to prune infeasible task plans. However, these approaches only add constraints upon refinement failure, expending significant search effort on infeasible branches. We propose VIZ-COAST, a method of leveraging the common-sense spatial reasoning of large pretrained Vision-Language Models to identify issues with downward refinement a priori, bypassing the need to fix these failures during planning. Experiments on three challenging TAMP domains show that our approach is able to extract plausible constraints from images and domain descriptions, drastically reducing planning times and, in some cases, eliminating downward refinement failures altogether, generalizing to a diverse range of instances from the broader domain.