mlsanityData to 14 Oct 2026

Retrieval-Augmented Code Generation for Situated Action Generation: A Case Study on Minecraft

arXiv 2406.17553 ↗First posted 25 Jun 2024cs.CLcited by 4

On the map: Retrieval-augmented generation → RAG for code

Abstract (arXiv)

In the Minecraft Collaborative Building Task, two players collaborate: an Architect (A) provides instructions to a Builder (B) to assemble a specified structure using 3D blocks. In this work, we investigate the use of large language models (LLMs) to predict the sequence of actions taken by the Builder. Leveraging LLMs' in-context learning abilities, we use few-shot prompting techniques, that significantly improve performance over baseline methods. Additionally, we present a detailed analysis of the gaps in performance for future work

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in RAG for code, one of 15 directions in Retrieval-augmented generation. Builds on 21 of its 46 references that are in the map. First posted on arXiv 25 Jun 2024.

Builds on

21 of 46 references.

References · 21

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  7. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  8. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  9. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  10. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  11. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
  12. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
9 more
  1. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
  2. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 723
  3. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  4. 2023ChatGPT for Robotics: Design Principles and Model Abilitiescited by 299
  5. 2022Learning to Execute Actions or Ask Clarification Questionscited by 30
  6. 2023Transforming Human-Centered AI Collaboration: Redefining Embodied Agents Capabilities through Interactive Grounded Language Instructionscited by 13
  7. 2024Your Co-Workers Matter: Evaluating Collaborative Capabilities of Language Models in Blocks Worldcited by 9
  8. 2024Large Language Models as Minecraft Agentscited by 7
  9. 2023Symbolic Planning and Code Generation for Grounded Dialoguecited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2025Retrieval-Augmented Code Generation: A Survey with Focus on Repository-Level Approachescited by 23
  2. 2024A LLM Benchmark based on the Minecraft Builder Dialog Agent Taskcited by 4
  3. 2025GuessBench: Sensemaking Multimodal Creativity in the Wildcited by 1
  4. 2026MineCEraft: Evaluating Language Models as Construction Engineers in the World of Minecraftcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025BAP v2: An Enhanced Task Framework for Instruction Following in Minecraft Dialogues
  2. 2024Improving Agent Interactions in Virtual Environments with Language Models
  3. 2024APT: Architectural Planning and Text-to-Blueprint Construction Using Large Language Models for Open-World Agents
  4. 2023See and Think: Embodied Agent in Virtual Environment
  5. 2025From Entity-Centric to Goal-Oriented Graphs: Enhancing LLM Knowledge Retrieval in Minecraft
  6. 2025Collaborating Action by Action: A Multi-agent LLM Framework for Embodied Reasoning
  7. 2025JARVIS-VLA: Post-Training Large-Scale Vision Language Models to Play Visual Games with Keyboards and Mouse
  8. 2023STEVE-1: A Generative Model for Text-to-Behavior in Minecraft