mlsanityData to 14 Oct 2026

Using Language Models as Closed-Loop High-Level Planners for Robotics Applications: A Brief Overview and Benchmarks

arXiv 2511.07410 ↗First posted 10 Nov 2025cs.RO cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Large Language Models (LLMs) and Vision Language Models (VLMs) have become popular tools for embodied high-level planning. However, their deployment in black-box settings often leads to unpredictable or costly errors. To harness their capabilities more reliably in robotic systems, we empirically investigate practical strategies for integrating language models as closed-loop planners. Concretely, we study how the control horizon and warm-starting impact the performance of language model-based planners. We design and conduct controlled experiments to extract actionable insights, providing recommendations that can help improve the performance and robustness of language model-based embodied planning. The full implementation and experiments are available on the project website

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 24 of its 33 references that are in the map. First posted on arXiv 10 Nov 2025.

Builds on

24 of 33 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Reflective Planning: Vision-Language Models for Multi-Stage Long-Horizon Robotic Manipulation23 Feb
  2. 2024Guiding Long-Horizon Task and Motion Planning with Vision Language Models3 Oct
  3. 2024ReplanVLM: Replanning Robotic Tasks with Visual Language Models31 Jul
  4. 2024Robotic Control via Embodied Chain-of-Thought Reasoning11 Jul
  5. 2023Grounding Classical Task Planners via Vision-Language Models17 Apr
  6. 2023PaLM-E: An Embodied Multimodal Language Model6 Mar

Other references · 18

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  5. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  6. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
  7. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
  8. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  9. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 611
  10. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 501
  11. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 462
  12. 2025Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Modelscited by 221
6 more
  1. 2023Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planningcited by 161
  2. 2023Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language Modelcited by 116
  3. 2023AutoTAMP: Autoregressive Task and Motion Planning with LLMs as Translators and Checkerscited by 113
  4. 2024LLM3:Large Language Model-based Task and Motion Planning with Motion Failure Reasoningcited by 63
  5. 2024RePLan: Robotic Replanning with Perception and Language Modelscited by 41
  6. 2024Socratic Planner: Self-QA-Based Zero-Shot Planning for Embodied Instruction Followingcited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Can LLM-Reasoning Models Replace Classical Planning? A Benchmark Study
  2. 2025Constrained Natural Language Action Planning for Resilient Embodied Systems
  3. 2024DaDu-E: Rethinking the Role of Large Language Model in Robotic Computing Pipeline
  4. 2025A Systematic Study of Large Language Models for Task and Motion Planning With PDDLStream
  5. 2024Large Language Models for Robotics: Opportunities, Challenges, and Perspectives
  6. 2025Making VLMs More Robot-Friendly: Self-Critical Distillation of Low-Level Procedural Reasoning
  7. 2025Assessing the Value of Visual Input: A Benchmark of Multimodal Large Language Models for Robotic Path Planning
  8. 2025ExploreVLM: Closed-Loop Robot Exploration Task Planning with Vision-Language Models