mlsanityData to 7 Oct 2026

Large Language Models as Minecraft Agents

arXiv 2402.08392 ↗First posted 13 Feb 2024cs.CLcited by 7

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

In this work we examine the use of Large Language Models (LLMs) in the challenging setting of acting as a Minecraft agent. We apply and evaluate LLMs in the builder and architect settings, introduce clarification questions and examining the challenges and opportunities for improvement. In addition, we present a platform for online interaction with the agents and an evaluation against previous works.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 22 of its 39 references that are in the map. First posted on arXiv 13 Feb 2024.

Builds on

22 of 39 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory25 May
  2. 2023Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modelling28 Jan

Other references · 20

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  6. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  7. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  8. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  9. 2019Learning to Speak and Act in a Fantasy Text Adventure Gamecited by 122
  10. 2023AgentSims: An Open-Source Sandbox for Large Language Model Evaluationcited by 86
  11. 2023Towards autonomous system: flexible modular production system enhanced with large language model agentscited by 38
  12. 2022Dungeons and Dragons as a Dialog Challenge for Artificial Intelligencecited by 35
8 more
  1. 2023Improving Grounded Language Understanding in a Collaborative Environment by Interacting with Agents Through Help Feedbackcited by 31
  2. 2022Learning to Execute Actions or Ask Clarification Questionscited by 30
  3. 2019CraftAssist: A Framework for Dialogue-enabled Interactive Agentscited by 27
  4. 2022Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 2021cited by 23
  5. 2019Why Build an Assistant in Minecraft?cited by 14
  6. 2023Transforming Human-Centered AI Collaboration: Redefining Embodied Agents Capabilities through Interactive Grounded Language Instructionscited by 13
  7. 2022Collecting Interactive Multi-modal Datasets for Grounded Language Understandingcited by 10
  8. 2022Learning to Solve Voxel Building Embodied Tasks from Pixels and Natural Language Instructionscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 0; 2026: 2.

7 papers.

  1. 2024Language Models are Crossword Solverscited by 5
  2. 2024Retrieval-Augmented Code Generation for Situated Action Generation: A Case Study on Minecraftcited by 4
  3. 2024A LLM Benchmark based on the Minecraft Builder Dialog Agent Taskcited by 4
  4. 2024Nebula: A discourse aware Minecraft Buildercited by 3
  5. 2024Towards No-Code Programming of Cobots: Experiments with Code Synthesis by Large Code Models for Conversational Programmingcited by 2
  6. 2026Mixing Expert Knowledge: Bring Human Thoughts Back To the Game of Gocited by 1
  7. 2026Correct Answers from Sound Reasoning: Verifiable Process Supervision for Language Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024STEVE Series: Step-by-Step Construction of Agent Systems in Minecraft
  2. 2023AgentBench: Evaluating LLMs as Agents
  3. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges
  4. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challenges
  5. 2023Clembench: Using Game Play to Evaluate Chat-Optimized Language Models as Conversational Agents
  6. 2023See and Think: Embodied Agent in Virtual Environment
  7. 2024Large Language Model-Based Agents for Software Engineering: A Survey
  8. 2024A Survey on Large Language Model-Based Game Agents