mlsanityData to 7 Oct 2026

Retrospective on the 2021 BASALT Competition on Learning from Human Feedback

arXiv 2204.07123 ↗First posted 14 Apr 2022cs.AIcited by 5

On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF

Abstract (arXiv)

We held the first-ever MineRL Benchmark for Agents that Solve Almost-Lifelike Tasks (MineRL BASALT) Competition at the Thirty-fifth Conference on Neural Information Processing Systems (NeurIPS 2021). The goal of the competition was to promote research towards agents that use learning from human feedback (LfHF) techniques to solve open-world tasks. Rather than mandating the use of LfHF techniques, we described four tasks in natural language to be accomplished in the video game Minecraft, and allowed participants to use any approach they wanted to build agents that could accomplish the tasks. Teams developed a diverse range of LfHF algorithms across a variety of possible human feedback types. The three winning teams implemented significantly different approaches while achieving similar performance. Interestingly, their approaches performed well on different tasks, validating our choice of tasks to include in the competition. While the outcomes validated the design of our competition, we did not get as many participants and submissions as our sister competition, MineRL Diamond. We speculate about the causes of this problem and suggest improvements for future iterations of the competition.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 14 of its 27 references that are in the map. First posted on arXiv 14 Apr 2022.

Builds on

14 of 27 references.

Closest ancestors · 3

Same direction, newest first

  1. 2021Combining Learning from Human Feedback and Knowledge Engineering to Solve Hierarchical Tasks in Minecraft7 Dec
  2. 2018Reward learning from human preferences and demonstrations in Atari15 Nov
  3. 2017Deep reinforcement learning from human preferences12 Jun

Other references · 11

  1. 2016Generative Adversarial Imitation Learningcited by 1,576
  2. 2015Prioritized Experience Replaycited by 1,070
  3. 2017Inverse Reward Designcited by 246
  4. 2021IQ-Learn: Inverse soft-Q Learning for Imitationcited by 191
  5. 2020Reward-rational (implicit) choice: A unifying formalism for reward learningcited by 119
  6. 2019Preferences Implicit in the State of the Worldcited by 33
  7. 2019CraftAssist: A Framework for Dialogue-enabled Interactive Agentscited by 27
  8. 2019The MineRL 2019 Competition on Sample Efficient Reinforcement Learning using Human Priorscited by 25
  9. 2021The MineRL BASALT Competition on Learning from Human Feedbackcited by 24
  10. 2018Generative Design in Minecraft (GDMC), Settlement Generation Competitioncited by 22
  11. 2019The Multi-Agent Reinforcement Learning in Malm"O (MARL"O) Competitioncited by 14

Built on by

Cited by year

Citing papers per year: 2023: 5.

5 papers.

  1. 2023Towards Solving Fuzzy Tasks with Human Feedback: A Retrospective of the MineRL BASALT 2022 Competitioncited by 15
  2. 2023Reinforcement Learning Friendly Vision-Language Model for Minecraftcited by 14
  3. 2023BEDD: The MineRL BASALT Evaluation and Demonstrations Dataset for Training and Benchmarking Agents that Solve Fuzzy Taskscited by 8
  4. 2023DIP-RL: Demonstration-Inferred Preference Learning in Minecraftcited by 2
  5. 2023Benchmarking Robustness and Generalization in Multi-Agent Systems: A Case Study on Neural MMOcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020Retrospective Analysis of the 2019 MineRL Competition on Sample Efficient Reinforcement Learning
  2. 2020Playing Minecraft with Behavioural Cloning
  3. 2024TeamCraft: A Benchmark for Multi-Modal Multi-Agent Systems in Minecraft
  4. 2019Deep Reinforcement Learning from Policy-Dependent Human Feedback
  5. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge
  6. 2025BAP v2: An Enhanced Task Framework for Instruction Following in Minecraft Dialogues
  7. 2024Uni-RLHF: Universal Platform and Benchmark Suite for Reinforcement Learning with Diverse Human Feedback
  8. 2024Odyssey: Empowering Minecraft Agents with Open-World Skills