mlsanityData to 7 Oct 2026

RL-GPT: Integrating Reinforcement Learning and Code-as-policy

arXiv 2402.19299 ↗First posted 29 Feb 2024cs.AI cs.LGcited by 32

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated proficiency in utilizing various tools by coding, yet they face limitations in handling intricate logic and precise control. In embodied tasks, high-level planning is amenable to direct coding, while low-level actions often necessitate task-specific refinement, such as Reinforcement Learning (RL). To seamlessly integrate both modalities, we introduce a two-level hierarchical framework, RL-GPT, comprising a slow agent and a fast agent. The slow agent analyzes actions suitable for coding, while the fast agent executes coding tasks. This decomposition effectively focuses each agent on specific tasks, proving highly efficient within our pipeline. Our approach outperforms traditional RL methods and existing GPT agents, demonstrating superior efficiency. In the Minecraft game, it rapidly obtains diamonds within a single day on an RTX3090. Additionally, it achieves SOTA performance across all designated MineDojo tasks.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 55 of its 72 references that are in the map. First posted on arXiv 29 Feb 2024.

Builds on

55 of 72 references.

References · 55

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  7. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  8. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  9. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  10. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  11. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  12. 2023Mastering Diverse Domains through World Modelscited by 1,193
38 more
  1. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  2. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  3. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  4. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  5. 2023Large Language Models as Optimizerscited by 830
  6. 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
  7. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
  8. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  9. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  10. 2023AppAgent: Multimodal Agents as Smartphone Userscited by 407
  11. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  12. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 396
  13. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 380
  14. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 352
  15. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 350
  16. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 338
  17. 2023Language to Rewards for Robotic Skill Synthesiscited by 307
  18. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memorycited by 288
  19. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 220
  20. 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 210
  21. 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 193
  22. 2018Programmatically Interpretable Reinforcement Learningcited by 192
  23. 2018Verifiable Reinforcement Learning via Policy Extractioncited by 176
  24. 2023JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Modelscited by 170
  25. 2024RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedbackcited by 147
  26. 2023STEVE-1: A Generative Model for Text-to-Behavior in Minecraftcited by 95
  27. 2023Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modellingcited by 88
  28. 2023Bootstrap Your Own Skills: Learning to Solve New Tasks with Large Language Model Guidancecited by 85
  29. 2023Octopus: Embodied Vision-Language Programmer from Environmental Feedbackcited by 73
  30. 2019IKEA Furniture Assembly Environment for Long-Horizon Complex Manipulation Taskscited by 68
  31. 2023Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraftcited by 60
  32. 2021Learning to Synthesize Programs as Interpretable and Generalizable Policiescited by 59
  33. 2022Robotic Skill Acquisition via Instruction Augmentation with Vision-Language Modelscited by 58
  34. 2023DERA: Enhancing Large Language Model Completions with Dialog-Enabled Resolving Agentscited by 52
  35. 2023Steve-Eye: Equipping LLM-based Embodied Agents with Visual Perception in Open Worldscited by 45
  36. 2023Query-Dependent Prompt Evaluation and Optimization with Offline Inverse RLcited by 37
  37. 2023LLM as OS, Agents as Apps: Envisioning AIOS, Agents and the AIOS-Agent Ecosystemcited by 35
  38. 2022MineRL Diamond 2021 Competition: Overview, Results, and Lessons Learnedcited by 26

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 13; 2026: 11.

32 papers.

Sort:

By area:

  1. 2024A Survey on Large Language Model-Based Game Agentscited by 106
  2. 2025A Survey on Vision-Language-Action Models: An Action Tokenization Perspectivecited by 96
  3. 2025Monte Carlo Planning with Large Language Model for Text-Based Game Agentscited by 22
  4. 2025Being-0: A Humanoid Robotic Agent with Vision-Language Models and Modular Skillscited by 21
  5. 2024ROCKET-1: Mastering Open-World Interaction with Visual-Temporal Context Promptingcited by 20
  6. 2024FASIONAD : FAst and Slow FusION Thinking Systems for Human-Like Autonomous Driving with Adaptive Feedbackcited by 18
  7. 2025Optimus-3: Dual-Router Aligned Mixture-of-Experts Agent with Dual-Granularity Reasoning-Aware Policy Optimizationcited by 12
  8. 2024Stop Wandering, Find the Keys: LLMs Discriminate Key States for Efficient Multi-Agent Explorationcited by 9
  9. 2025FOUNDER: Grounding Foundation Models in World Models for Open-Ended Embodied Decision Makingcited by 8
  10. 2025AnimeGamer: Infinite Anime Life Simulation with Next Game State Predictioncited by 7
  11. 2026Evolving Programmatic Skill Networkscited by 7
  12. 2024Zero-shot Model-based Reinforcement Learning using Large Language Modelscited by 6
20 more
  1. 2024The Duality of Generative AI and Reinforcement Learning in Robotics: A Reviewcited by 5
  2. 2025Think Twice, Act Once: A Co-Evolution Framework of LLM and RL for Large-Scale Decision Makingcited by 5
  3. 2026NOVA: NOise-aware Verbal Confidence CAlibration for Robust Large Language Models in RAG Systemscited by 5
  4. 2026SCALAR: Learning and Composing Skills through LLM Guided Symbolic Planning and Deep RL Groundingcited by 4
  5. 2026Experience Transfer for Multimodal LLM Agents in Minecraft Gamecited by 4
  6. 2026TensorHub: Scalable and Elastic Weight Transfer for LLM RL Trainingcited by 4
  7. 2024Language-Model-Assisted Bi-Level Programming for Reward Learning from Internet Videoscited by 3
  8. 2025Scalable Multi-Task Reinforcement Learning for Generalizable Spatial Intelligence in Visuomotor Agentscited by 3
  9. 2026MEMENTO: Memory-Guided Memetic Code-as-Policy Evolutioncited by 3
  10. 2025Dynamic Optimizations of LLM Ensembles with Two-Stage Reinforcement Learning Agentscited by 2
  11. 2025Cultivating Game Sense for Yourself: Making VLMs Gaming Expertscited by 1
  12. 2026ReCAPA: Hierarchical Predictive Correction to Mitigate Cascading Failurescited by 1
  13. 2026LLM-PDESR: Robust PDE Discovery via Subdomain Weighted Residuals and LLM-Guided Symbolic Hypothesis Generationcited by 1
  14. 2024Role-RL: Online Long-Context Processing with Role Reinforcement Learning for Distinct LLMs in Their Optimal Rolescited by 0
  15. 2025BAR: A Backward Reasoning based Agent for Complex Minecraft Taskscited by 0
  16. 2025LinguaFluid: Language Guided Fluid Control via Semantic Rewards in Reinforcement Learningcited by 0
  17. 2025Policy-Conditioned Policies for Multi-Agent Task Solvingcited by 0
  18. 2026Near-Constant Strong Violation and Last-Iterate Convergence for Online CMDPs via Decaying Safety Marginscited by 0
  19. 2026Taming Asynchronous CPU-GPU Coupling for Frequency-aware Latency Estimation on Mobile Edgecited by 0
  20. 2026WeightBridge: An Efficient Weight Transfer Library for Reinforcement Learningcited by 0