mlsanityData to 14 Oct 2026

Towards Solving Fuzzy Tasks with Human Feedback: A Retrospective of the MineRL BASALT 2022 Competition

arXiv 2303.13512 ↗First posted 23 Mar 2023cs.AIcited by 15

On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF

Abstract (arXiv)

To facilitate research in the direction of fine-tuning foundation models from human feedback, we held the MineRL BASALT Competition on Fine-Tuning from Human Feedback at NeurIPS 2022. The BASALT challenge asks teams to compete to develop algorithms to solve tasks with hard-to-specify reward functions in Minecraft. Through this competition, we aimed to promote the development of algorithms that use human feedback as channels to learn the desired behavior. We describe the competition and provide an overview of the top solutions. We conclude by discussing the impact of the competition and future directions for improvement.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 25 of its 34 references that are in the map. First posted on arXiv 23 Mar 2023.

Builds on

25 of 34 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Retrospective on the 2021 BASALT Competition on Learning from Human Feedback14 Apr
  2. 2021WebGPT: Browser-assisted question-answering with human feedback17 Dec
  3. 2017Deep reinforcement learning from human preferences12 Jun

Other references · 22

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  6. 2019Searching for MobileNetV3cited by 1,994
  7. 2016OpenAI Gymcited by 1,736
  8. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 478
  9. 2016The Option-Critic Architecturecited by 448
  10. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 345
  11. 2021Benchmarking the Spectrum of Agent Capabilitiescited by 200
  12. 2021IQ-Learn: Inverse soft-Q Learning for Imitationcited by 193
10 more
  1. 2022imitation: Clean Imitation Learning Implementationscited by 34
  2. 2019CraftAssist: A Framework for Dialogue-enabled Interactive Agentscited by 28
  3. 2019The MineRL 2019 Competition on Sample Efficient Reinforcement Learning using Human Priorscited by 26
  4. 2021The MineRL BASALT Competition on Learning from Human Feedbackcited by 24
  5. 2022UniMASK: Unified Inference in Sequential Decision Problemscited by 24
  6. 2020EvoCraft: A New Challenge for Open-Endednesscited by 23
  7. 2018Generative Design in Minecraft (GDMC), Settlement Generation Competitioncited by 22
  8. 2021NeurIPS 2021 Competition IGLU: Interactive Grounded Language Understanding in a Collaborative Environmentcited by 15
  9. 2019The Multi-Agent Reinforcement Learning in Malm"O (MARL"O) Competitioncited by 14
  10. 2022Behavioral Cloning via Search in Video PreTraining Latent Spacecited by 9

Built on by

Cited by year

Citing papers per year: 2023: 6; 2024: 8; 2025: 1.

15 papers.

Sort:

By area:

  1. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memorycited by 289
  2. 2024Large Language Models and Games: A Survey and Roadmapcited by 93
  3. 2023Creative Agents: Empowering Agents with Imagination for Creative Taskscited by 24
  4. 2024Zero-shot Imitation Policy via Search in Demonstration Datasetcited by 13
  5. 2025AssistanceZero: Scalably Solving Assistance Gamescited by 10
  6. 2023BEDD: The MineRL BASALT Evaluation and Demonstrations Dataset for Training and Benchmarking Agents that Solve Fuzzy Taskscited by 8
  7. 2023Benchmarks for Physical Reasoning AIcited by 8
  8. 2024Natural Language as Policies: Reasoning for Coordinate-Level Embodied Control with LLMscited by 4
  9. 2024Interpretability in Action: Exploratory Analysis of VPT, a Minecraft Agentcited by 4
  10. 2023Behavioral Cloning via Search in Embedded Demonstration Datasetcited by 3
  11. 2023DIP-RL: Demonstration-Inferred Preference Learning in Minecraftcited by 2
  12. 2024Aligning Agents like Large Language Modelscited by 1
3 more
  1. 2024SOAP-RL: Sequential Option Advantage Propagation for Reinforcement Learning in POMDP Environmentscited by 1
  2. 2024Object and Contact Point Tracking in Demonstrations Using 3D Gaussian Splattingcited by 0
  3. 2024STEVE-Audio: Expanding the Goal Conditioning Modalities of Embodied Agents in Minecraftcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Combining Learning from Human Feedback and Knowledge Engineering to Solve Hierarchical Tasks in Minecraft
  2. 2020Retrospective Analysis of the 2019 MineRL Competition on Sample Efficient Reinforcement Learning
  3. 2020Playing Minecraft with Behavioural Cloning
  4. 2021JueWu-MC: Playing Minecraft with Sample-efficient Hierarchical Reinforcement Learning
  5. 2025BAP v2: An Enhanced Task Framework for Instruction Following in Minecraft Dialogues
  6. 2024TeamCraft: A Benchmark for Multi-Modal Multi-Agent Systems in Minecraft
  7. 2025Results of the NeurIPS 2023 Neural MMO Competition on Multi-task Reinforcement Learning
  8. 2025Matrix-Game: Interactive World Foundation Model