mlsanityData to 7 Oct 2026

ChatGPT Empowered Long-Step Robot Control in Various Environments: A Case Application

arXiv 2304.03893 ↗First posted 8 Apr 2023cs.ROcited by 48

Not on the map. Why?

Abstract (arXiv)

This paper demonstrates how OpenAI's ChatGPT can be used in a few-shot setting to convert natural language instructions into a sequence of executable robot actions. The paper proposes easy-to-customize input prompts for ChatGPT that meet common requirements in practical applications, such as easy integration with robot execution systems and applicability to various environments while minimizing the impact of ChatGPT's token limit. The prompts encourage ChatGPT to output a sequence of predefined robot actions, represent the operating environment in a formalized style, and infer the updated state of the operating environment. Experiments confirmed that the proposed prompts enable ChatGPT to act according to requirements in various environments, and users can adjust ChatGPT's output with natural language feedback for safe and robust operation. The proposed prompts and source code are open-source and publicly available at https://github.com/microsoft/ChatGPT-Robot-Manipulation-Prompts

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Apr 2023.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 16; 2024: 19; 2025: 12; 2026: 1.

48 papers.

Sort:

By area:

  1. 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 289
  2. 2023AutoTAMP: Autoregressive Task and Motion Planning with LLMs as Translators and Checkerscited by 111
  3. 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 109
  4. 2024Large Language Models for Robotics: Opportunities, Challenges, and Perspectivescited by 88
  5. 2023GPT-4V(ision) for Robotics: Multimodal Task Planning from Human Demonstrationcited by 76
  6. 2024Real-World Robot Applications of Foundation Models: A Reviewcited by 60
  7. 2023TPTU: Large Language Model-based AI Agents for Task Planning and Tool Usagecited by 54
  8. 2024AutoManual: Constructing Instruction Manuals by LLM Agents via Interactive Environmental Learningcited by 52
  9. 2023CoPAL: Corrective Planning of Robot Actions with Large Language Modelscited by 39
  10. 2023TPTU-v2: Boosting Task Planning and Tool Usage of Large Language Model-based Agents in Real-world Systemscited by 39
  11. 2023Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMscited by 35
  12. 2023LLM as A Robotic Brain: Unifying Egocentric Memory and Controlcited by 32
36 more
  1. 2024HAZARD Challenge: Embodied Decision Making in Dynamically Changing Environmentscited by 32
  2. 2024An Interactive Agent Foundation Modelcited by 25
  3. 2023Incremental Learning of Humanoid Robot Behavior from Natural Interaction and Large Language Modelscited by 19
  4. 2023Decoding ChatGPT: A Taxonomy of Existing Research, Current Challenges, and Possible Future Directionscited by 18
  5. 2024AutoGPT+P: Affordance-based Task Planning with Large Language Modelscited by 17
  6. 2023InterAct: Exploring the Potentials of ChatGPT as a Cooperative Agentcited by 16
  7. 2025Learning Compositional Behaviors from Demonstration and Languagecited by 15
  8. 2023A Safer Vision-based Autonomous Planning System for Quadrotor UAVs with Dynamic Obstacle Trajectory Prediction and Its Application with LLMscited by 13
  9. 2024LLMs with Industrial Lens: Deciphering the Challenges and Prospects -- A Surveycited by 12
  10. 2024RLingua: Improving Reinforcement Learning Sample Efficiency in Robotic Manipulations With Large Language Modelscited by 11
  11. 2025Code-as-Symbolic-Planner: Foundation Model-Based Robot Planning via Symbolic Code Generationcited by 10
  12. 2023Selective Perception: Optimizing State Descriptions with Reinforcement Learning for Language Model Actorscited by 9
  13. 2024Don't Let Your Robot be Harmful: Responsible Robotic Manipulation via Safety-as-Policycited by 9
  14. 2024AlignBot: Aligning VLM-powered Customized Task Planning with User Reminders Through Fine-Tuning for Household Robotscited by 7
  15. 2025Structured Preference Optimization for Vision-Language Long-Horizon Task Planningcited by 7
  16. 2023WALL-E: Embodied Robotic WAiter Load Lifting with Large Language Modelcited by 6
  17. 2024Polaris: Open-ended Interactive Robotic Manipulation via Syn2Real Visual Grounding and Large Language Modelscited by 6
  18. 2024Open-Vocabulary Action Localization with Iterative Visual Promptingcited by 5
  19. 2023GPT Models Meet Robotic Applications: Co-Speech Gesturing Chat Systemcited by 4
  20. 2024MEIA: Multimodal Embodied Perception and Interaction in Unknown Environmentscited by 4
  21. 2024LIMT: Language-Informed Multi-Task Visual World Modelscited by 4
  22. 2024VernaCopter: Disambiguated Natural-Language-Driven Robot via Formal Specificationscited by 4
  23. 2025Instruction-Augmented Long-Horizon Planning: Embedding Grounding Mechanisms in Embodied Mobile Manipulationcited by 4
  24. 2025MUSON: A Reasoning-oriented Multimodal Dataset for Socially Compliant Navigation in Urban Environmentscited by 4
  25. 2024No Size Fits All: The Perils and Pitfalls of Leveraging LLMs Vary with Company Sizecited by 3
  26. 2025Reasoner for Real-World Event Detection: Scaling Reinforcement Learning via Adaptive Perplexity-Aware Sampling Strategycited by 3
  27. 2024A Roadmap Towards Automated and Regulated Robotic Systemscited by 2
  28. 2024Exploring Gen-AI applications in building research and industry: A reviewcited by 2
  29. 2024Temporal and Semantic Evaluation Metrics for Foundation Models in Post-Hoc Analysis of Robotic Sub-taskscited by 1
  30. 2025Plan-and-Act using Large Language Models for Interactive Agreementcited by 1
  31. 2025Reducing Latency in LLM-Based Natural Language Commands Processing for Robot Navigationcited by 1
  32. 2025RoboPARA: Dual-Arm Robot Planning with Parallel Allocation and Recomposition Across Taskscited by 1
  33. 2025SmallPlan: Leverage Small Language Models for Sequential Path Planning with Simulation-Powered, LLM-Guided Distillationcited by 0
  34. 2025Toward Accurate Long-Horizon Robotic Manipulation: Language-to-Action with Foundation Models via Scene Graphscited by 0
  35. 2025HELP: Hierarchical Embodied Language Planner for Household Taskscited by 0
  36. 2026AgentRob: From Virtual Forum Agents to Hijacked Physical Robotscited by 0