RoboPilot: Generalizable Dynamic Robotic Manipulation with Dual-thinking Modes
On the map: LLM reasoning → Spatial and embodied reasoning
Abstract (arXiv)
Despite rapid progress in robotics, complex or long-horizon tasks remain a fundamental challenge. Most current approaches follow an open-loop paradigm with limited reasoning and no feedback, resulting in poor robustness to environmental changes and severe error accumulation. We present RoboPilot, a dual-thinking closed-loop agentic framework for robotic manipulation that supports adaptive reasoning for complex tasks in real-world dynamic environments. RoboPilot leverages primitive actions for structured task planning and flexible action generation as a agentic system, while introducing feedback to enable replanning from dynamic changes and execution errors. Chain-of-Thought reasoning further enhances high-level task planning and guides low-level action generation. The agentic system dynamically switches between fast and slow thinking to balance efficiency and accuracy. To systematically evaluate the robustness of RoboPilot in diverse robot manipulation scenarios, we introduce RoboPilot-Bench, a benchmark spanning 21 tasks across 10 categories, including infeasible-task recognition and dynamic recovery. Experiments show that RoboPilot outperforms state-of-the-art baselines by 11% in task success rate, and the real-world deployment on an industrial robot further demonstrates its robustness.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 25 of its 31 references that are in the map. First posted on arXiv 30 Sep 2025.
Builds on
25 of 31 references.
Closest ancestors · 3
Same direction, newest first
- 2024Robotic Control via Embodied Chain-of-Thought Reasoning11 Jul
- 2024Learning Manipulation Skills through Robot Chain-of-Thought with Sparse Failure Guidance22 May
- 2024Language-Conditioned Robotic Manipulation with Fast and Slow Thinking8 Jan
Other references · 22
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2024GPT-4o System Cardcited by 4,467
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
- 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
- 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
- 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
- 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
- 2022Visual Programming: Compositional visual reasoning without trainingcited by 507
- 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
- 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 289
10 more
- 2023REFLECT: Summarizing Robot Experiences for Failure Explanation and Correctioncited by 171
- 2023Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language Modelcited by 115
- 2024Large Language Models for Robotics: Opportunities, Challenges, and Perspectivescited by 88
- 2024FMB: a Functional Manipulation Benchmark for Generalizable Robotic Learningcited by 79
- 2025MuJoCo Playgroundcited by 57
- 2023RobotGPT: Robot Manipulation Learning from ChatGPTcited by 39
- 2024DynaMem: Online Dynamic Spatio-Semantic Memory for Open World Mobile Manipulationcited by 32
- 2024InterPreT: Interactive Predicate Learning from Language Feedback for Generalizable Task Planningcited by 31
- 2025Generative Artificial Intelligence in Robotic Manipulation: A Surveycited by 16
- 2024Open-vocabulary Mobile Manipulation in Unseen Dynamic Environments with 3D Semantic Mapscited by 13
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026ManipArena: Comprehensive Real-world Evaluation of Reasoning-Oriented Generalist Robot Manipulation
- 2026IMBench: A Benchmark for Intuitive Robotic Manipulation
- 2026ManipulationNet: An Infrastructure for Benchmarking Real-World Robot Manipulation with Physical Skill Challenges and Embodied Multimodal Reasoning
- 2025RoBridge: A Hierarchical Architecture Bridging Cognition and Execution for General Robotic Manipulation
- 2025RoboCerebra: A Large-scale Benchmark for Long-horizon Robotic Manipulation Evaluation
- 2024Logic Learning from Demonstrations for Multi-step Manipulation Tasks in Dynamic Environments
- 2026RoboWits: Unexpected Challenges for Robotic Creative Problem Solving
- 2026RoboHarn-Evo: Evolving Hierarchical Physical Knowledge for Self-Improving Robotic Manipulation