mlsanityData to 7 Oct 2026

RobotGPT: Robot Manipulation Learning from ChatGPT

arXiv 2312.01421 ↗First posted 3 Dec 2023cs.ROcited by 39

Not on the map. Why?

Abstract (arXiv)

We present RobotGPT, an innovative decision framework for robotic manipulation that prioritizes stability and safety. The execution code generated by ChatGPT cannot guarantee the stability and safety of the system. ChatGPT may provide different answers for the same task, leading to unpredictability. This instability prevents the direct integration of ChatGPT into the robot manipulation loop. Although setting the temperature to 0 can generate more consistent outputs, it may cause ChatGPT to lose diversity and creativity. Our objective is to leverage ChatGPT's problem-solving capabilities in robot manipulation and train a reliable agent. The framework includes an effective prompt structure and a robust learning model. Additionally, we introduce a metric for measuring task difficulty to evaluate ChatGPT's performance in robot manipulation. Furthermore, we evaluate RobotGPT in both simulation and real-world environments. Compared to directly using ChatGPT to generate code, our framework significantly improves task success rates, with an average increase from 38.5% to 91.5%. Therefore, training a RobotGPT by utilizing ChatGPT as an expert is a more stable approach compared to directly using ChatGPT as a task planner.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Dec 2023.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 14; 2026: 11.

39 papers.

Sort:

By area:

  1. 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulationcited by 34
  2. 2024Recover: A Neuro-Symbolic Framework for Failure Detection and Recoverycited by 15
  3. 2025RoboDexVLM: Visual Language Model-Enabled Task Planning and Motion Control for Dexterous Robot Manipulationcited by 15
  4. 2024P-RAG: Progressive Retrieval Augmented Generation For Planning on Embodied Everyday Taskcited by 14
  5. 2024Towards Open-World Grasping with Large Vision-Language Modelscited by 13
  6. 2024Robot Collapse: Supply Chain Backdoor Attacks Against VLM-based Robotic Manipulationcited by 13
  7. 2025Exploring Embodied Multimodal Large Models: Development, Datasets, and Future Directionscited by 12
  8. 2024LLMs for Coding and Robotics Educationcited by 9
  9. 2025Hierarchical Planning for Complex Tasks with Knowledge Graph-RAG and Symbolic Verificationcited by 9
  10. 2025Grounding Actions in Camera Space: Observation-Centric Vision-Language-Action Policycited by 8
  11. 2024Tell Me Where You Are: Multimodal LLMs Meet Place Recognitioncited by 7
  12. 2025Survey of Vision-Language-Action Models for Embodied Manipulationcited by 7
27 more
  1. 2024VernaCopter: Disambiguated Natural-Language-Driven Robot via Formal Specificationscited by 4
  2. 2024TalkWithMachines: Enhancing Human-Robot Interaction for Interpretable Industrial Robotics Through Large/Vision Language Modelscited by 4
  3. 2025GenSwarm: Scalable Multi-Robot Code-Policy Generation and Deployment via Language Modelscited by 4
  4. 2024Foundation Models for Autonomous Robots in Unstructured Environmentscited by 3
  5. 2025HybridGen: VLM-Guided Hybrid Planning for Scalable Data Generation of Imitation Learningcited by 3
  6. 2024The Future of Intelligent Healthcare: A Systematic Analysis and Discussion on the Integration and Impact of Robots Using Large Language Models for Healthcarecited by 2
  7. 2025OmniDexVLG: Learning Dexterous Grasp Generation from Vision Language Model-Guided Grasp Semantics, Taxonomy and Functional Affordancecited by 2
  8. 2024LLM-guided Task and Motion Planning using Knowledge-based Reasoningcited by 1
  9. 2025ImitDiff: Transferring Foundation-Model Priors for Distraction Robust Visuomotor Policycited by 1
  10. 2025Towards Autonomous Reinforcement Learning for Real-World Robotic Manipulation with Large Language Modelscited by 1
  11. 2025Boosting Robotic Manipulation Generalization with Minimal Costly Datacited by 1
  12. 2026Zero-shot HOI Detection with MLLM-based Detector-agnostic Interaction Recognitioncited by 1
  13. 2026SADP: Subgoal-Aware Diffusion Policy for Long-Horizon Manipulation Learned from Foundation Model Generated Demonstrationscited by 1
  14. 2024Human-centered In-building Embodied Delivery Benchmarkcited by 0
  15. 2024D-RMGPT: Robot-assisted collaborative tasks driven by large multimodal modelscited by 0
  16. 2025REFLEX: Metacognitive Reasoning for Reflective Zero-Shot Robotic Planning with Large Language Modelscited by 0
  17. 2025RoboPilot: Generalizable Dynamic Robotic Manipulation with Dual-thinking Modescited by 0
  18. 2025LLM-Cave: A benchmark and light environment for large language models reasoning and decision-making systemcited by 0
  19. 2026Zero-shot adaptable task planning for autonomous construction robots: a comparative study of lightweight single and multi-AI agent systemscited by 0
  20. 2026Ontology-Driven Robotic Specification Synthesiscited by 0
  21. 2026Bridging Speech, Emotion, and Motion: a VLM-based Multimodal Edge-deployable Framework for Humanoid Robotscited by 0
  22. 2026Instruct2Act: From Human Instruction to Actions Sequencing and Execution via Robot Action Network for Robotic Manipulationcited by 0
  23. 2026Efficient Soft Actor-Critic with LLM-Based Action-Level Guidance for Continuous Controlcited by 0
  24. 2026Hybrid LLM-based Intelligent Framework for Robot Task Schedulingcited by 0
  25. 2026Embodied GPT-5.1: Evidence of a World Model?cited by 0
  26. 2026Hierarchical Skill Retrieval for Data-Efficient Adaptation of Vision-Language-Action Modelscited by 0
  27. 2026Exploiting Vulnerabilities: Universal Adversarial Attacks on Vision-Language-Action Models in Roboticscited by 0