mlsanityData to 7 Oct 2026

Voyager: An Open-Ended Embodied Agent with Large Language Models

arXiv 2305.16291 ↗First posted 25 May 2023cs.AI cs.LGcited by 2,081

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

We introduce Voyager, the first LLM-powered embodied lifelong learning agent in Minecraft that continuously explores the world, acquires diverse skills, and makes novel discoveries without human intervention. Voyager consists of three key components: 1) an automatic curriculum that maximizes exploration, 2) an ever-growing skill library of executable code for storing and retrieving complex behaviors, and 3) a new iterative prompting mechanism that incorporates environment feedback, execution errors, and self-verification for program improvement. Voyager interacts with GPT-4 via blackbox queries, which bypasses the need for model parameter fine-tuning. The skills developed by Voyager are temporally extended, interpretable, and compositional, which compounds the agent's abilities rapidly and alleviates catastrophic forgetting. Empirically, Voyager shows strong in-context lifelong learning capability and exhibits exceptional proficiency in playing Minecraft. It obtains 3.3x more unique items, travels 2.3x longer distances, and unlocks key tech tree milestones up to 15.3x faster than prior SOTA. Voyager is able to utilize the learned skill library in a new Minecraft world to solve novel tasks from scratch, while other techniques struggle to generalize. We open-source our full codebase and prompts at https://voyager.minedojo.org/.

Key facts. Cited by 2,081 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 70 of its 89 references that are in the map. First posted on arXiv 25 May 2023.

Builds on

70 of 89 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modelling28 Jan
  2. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge17 Jun

Other references · 68

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  7. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  8. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  11. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  12. 2015Continuous control with deep reinforcement learningcited by 3,279
38 more
  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  2. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  3. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  4. 2022Emergent Abilities of Large Language Modelscited by 2,483
  5. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  6. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  7. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  8. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  9. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  10. 2023Mastering Diverse Domains through World Modelscited by 1,193
  11. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  12. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  13. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  14. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  15. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  16. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  17. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  18. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  19. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
  20. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
  21. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  22. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  23. 2017A Brief Survey of Deep Reinforcement Learningcited by 466
  24. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  25. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 396
  26. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 342
  27. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 338
  28. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 260
  29. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 256
  30. 2023LEVER: Learning to Verify Language-to-Code Generation with Executioncited by 228
  31. 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 210
  32. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 207
  33. 2019Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutionscited by 182
  34. 2020DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learningcited by 159
  35. 2020Rearrangement: A Challenge for Embodied AIcited by 153
  36. 2023Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Modelscited by 152
  37. 2020iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenescited by 151
  38. 2020Automatic Curriculum Learning For Deep RL: A Short Surveycited by 141

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 180; 2024: 399; 2025: 405; 2026: 1097.

2,081 papers.

Sort:

By area:

  1. 2023Qwen Technical Reportcited by 3,273
  2. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  3. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  4. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  5. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  6. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  7. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  8. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
  9. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
  10. 2023Large Language Models as Optimizerscited by 830
  11. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  12. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
38 more
  1. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  2. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 580
  3. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
  4. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
  5. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  6. 2024Executable Code Actions Elicit Better LLM Agentscited by 537
  7. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 521
  8. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 514
  9. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 450
  10. 2025Group-in-Group Policy Optimization for LLM Agent Trainingcited by 442
  11. 2023Cognitive Architectures for Language Agentscited by 422
  12. 2024AFlow: Automating Agentic Workflow Generationcited by 395
  13. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agentscited by 385
  14. 2024Automated Design of Agentic Systemscited by 360
  15. 2023An Embodied Generalist Agent in 3D Worldcited by 355
  16. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 350
  17. 2024Agent Workflow Memorycited by 342
  18. 2023MLAgentBench: Evaluating Language Agents on Machine Learning Experimentationcited by 339
  19. 2023Character-LLM: A Trainable Agent for Role-Playingcited by 335
  20. 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 319
  21. 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 294
  22. 2023Large Language Models as Tool Makerscited by 293
  23. 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Taskscited by 289
  24. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
  25. 2023A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaborationcited by 281
  26. 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agentscited by 268
  27. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 257
  28. 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 253
  29. 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learningcited by 252
  30. 2023Challenges and Applications of Large Language Modelscited by 251
  31. 2023FireAct: Toward Language Agent Fine-tuningcited by 247
  32. 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 243
  33. 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 231
  34. 2024Data Interpreter: An LLM Agent For Data Sciencecited by 222
  35. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agentscited by 213
  36. 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 212
  37. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 206
  38. 2025AIDE: AI-Driven Exploration in the Space of Codecited by 206

50 most cited of 2,081 All 2,081 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025CrafterDojo: A Suite of Foundation Models for Building Open-Ended Embodied Agents in Crafter
  2. 2026MineExplorer: Evaluating Open-World Exploration of MLLM Agents in Minecraft
  3. 2026MineNPC-Task: Task Suite for Memory-Aware Minecraft Agents
  4. 2026PEAM: Parametric Embodied Agent Memory through Contrastive Internalization of Experience in Minecraft
  5. 2025Optimus-2: Multimodal Minecraft Agent with Goal-Observation-Action Conditioned Policy
  6. 2026CEDAR: Automata as Verifiable Interfaces for Language-Guided Embodied Action
  7. 2025Collaborating Action by Action: A Multi-agent LLM Framework for Embodied Reasoning
  8. 2021Combining Learning from Human Feedback and Knowledge Engineering to Solve Hierarchical Tasks in Minecraft