mlsanityData to 14 Oct 2026

Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraft

arXiv 2312.09238 ↗First posted 14 Dec 2023cs.AI cs.CL cs.CVcited by 61

Not on the map. Why?

Abstract (arXiv)

Many reinforcement learning environments (e.g., Minecraft) provide only sparse rewards that indicate task completion or failure with binary values. The challenge in exploration efficiency in such environments makes it difficult for reinforcement-learning-based agents to learn complex tasks. To address this, this paper introduces an advanced learning system, named Auto MC-Reward, that leverages Large Language Models (LLMs) to automatically design dense reward functions, thereby enhancing the learning efficiency. Auto MC-Reward consists of three important components: Reward Designer, Reward Critic, and Trajectory Analyzer. Given the environment information and task descriptions, the Reward Designer first design the reward function by coding an executable Python function with predefined observation inputs. Then, our Reward Critic will be responsible for verifying the code, checking whether the code is self-consistent and free of syntax and semantic errors. Further, the Trajectory Analyzer summarizes possible failure causes and provides refinement suggestions according to collected trajectories. In the next round, Reward Designer will further refine and iterate the dense reward function based on feedback. Experiments demonstrate a significant improvement in the success rate and learning efficiency of our agents in complex tasks in Minecraft, such as obtaining diamond with the efficient ability to avoid lava, and efficiently explore trees and animals that are sparse in the plains biome.

Key facts. Cited by 61 papers in the map (data through 14 Oct 2026). Builds on 41 of its 55 references that are in the map. First posted on arXiv 14 Dec 2023.

Builds on

41 of 55 references.

References · 41

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2023A Survey of Large Language Modelscited by 2,576
  6. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  7. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  8. 2023Mastering Diverse Domains through World Modelscited by 1,253
  9. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  10. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  11. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
  12. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
29 more
  1. 2022Towards Reasoning in Large Language Models: A Surveycited by 596
  2. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 571
  3. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 478
  4. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
  5. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 395
  6. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 345
  7. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
  8. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 327
  9. 2023Language to Rewards for Robotic Skill Synthesiscited by 312
  10. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memorycited by 289
  11. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 286
  12. 2018Large-Scale Study of Curiosity-Driven Learningcited by 280
  13. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 264
  14. 2023Reward Design with Language Modelscited by 261
  15. 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 215
  16. 2023LIV: Language-Image Representations and Rewards for Robotic Controlcited by 188
  17. 2023Guiding Pretraining in Reinforcement Learning with Large Language Modelscited by 187
  18. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 166
  19. 2023Vision-Language Models as Success Detectorscited by 142
  20. 2018Self-Imitation Learningcited by 138
  21. 2016A Deep Hierarchical Approach to Lifelong Learning in Minecraftcited by 90
  22. 2018Learning Navigation Behaviors End-to-End with AutoRLcited by 70
  23. 2022Inferring Rewards from Language in Contextcited by 50
  24. 2021Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraftcited by 35
  25. 2019Evolving Rewards to Automate Reinforcement Learningcited by 31
  26. 2021JueWu-MC: Playing Minecraft with Sample-efficient Hierarchical Reinforcement Learningcited by 28
  27. 2022EAGER: Asking and Answering Questions for Automatic Reward Shaping in Language-guided RLcited by 23
  28. 2021SEIHAI: A Sample-efficient Hierarchical AI for the MineRL Competitioncited by 20
  29. 2020Scaling Imitation Learning in Minecraftcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 24; 2025: 24; 2026: 13.

61 papers.

Sort:

By area:

  1. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 116
  2. 2024A Survey on Large Language Model-Based Game Agentscited by 112
  3. 2024A Systematic Survey on Large Language Models for Algorithm Designcited by 69
  4. 2024Multi-objective Evolution of Heuristic Using Large Language Modelcited by 54
  5. 2024EnvGen: Generating and Adapting Environments via LLMs for Training Embodied Agentscited by 46
  6. 2024Sparkle: Mastering Basic Spatial Capabilities in Vision Language Models Elicits Generalization to Spatial Reasoningcited by 34
  7. 2024RL-GPT: Integrating Reinforcement Learning and Code-as-policycited by 32
  8. 2024A Large Language Model-Driven Reward Design Framework via Dynamic Feedback for Reinforcement Learningcited by 30
  9. 2025A Comprehensive Survey on Multi-Agent Cooperative Decision-Making: Scenarios, Approaches, Challenges and Perspectivescited by 27
  10. 2024Odyssey: Empowering Minecraft Agents with Open-World Skillscited by 26
  11. 2025Optimus-2: Multimodal Minecraft Agent with Goal-Observation-Action Conditioned Policycited by 25
  12. 2024A Decision-Language Model (DLM) for Dynamic Restless Multi-Armed Bandit Tasks in Public Healthcited by 24
38 more
  1. 2025Game-TARS: Pretrained Foundation Models for Scalable Generalist Multimodal Game Agentscited by 24
  2. 2024iLLM-TSC: Integration reinforcement learning and large language model for traffic signal control policy improvementcited by 23
  3. 2024A Survey on Game Playing Agents and Large Models: Methods, Applications, and Challengescited by 22
  4. 2024CurricuLLM: Automatic Task Curricula Design for Learning Complex Robot Skills using Large Language Modelscited by 22
  5. 2024Continuously Improving Mobile Manipulation with Autonomous Real-World RLcited by 22
  6. 2025A Survey on Large Language Models for Automated Planningcited by 17
  7. 2024Open-World Reinforcement Learning over Long Short-Term Imaginationcited by 13
  8. 2025LAMARL: LLM-Aided Multi-Agent Reinforcement Learning for Cooperative Policy Generationcited by 13
  9. 2024Online Intrinsic Rewards for Decision Making Agents from Large Language Model Feedbackcited by 10
  10. 2024On the Modeling Capabilities of Large Language Models for Sequential Decision Makingcited by 9
  11. 2025CausalMACE: Causality Empowered Multi-Agents in Minecraft Cooperative Taskscited by 8
  12. 2025DSADF: Thinking Fast and Slow for Decision Makingcited by 7
  13. 2024Evaluating Vision-Language Models as Evaluators in Path Planningcited by 6
  14. 2025EscapeCraft: A 3D Room Escape Environment for Benchmarking Complex Multimodal Reasoning Abilitycited by 6
  15. 2026Experience Transfer for Multimodal LLM Agents in Minecraft Gamecited by 5
  16. 2024Language Models as Efficient Reward Function Searchers for Custom-Environment Multi-Objective Reinforcementcited by 4
  17. 2024Automated Rewards via LLM-Generated Progress Functionscited by 4
  18. 2024Large Language Model-Enhanced Reinforcement Learning for Generic Bus Holding Control Strategiescited by 4
  19. 2025LearningFlow: Automated Policy Learning Workflow for Urban Driving with Large Language Modelscited by 4
  20. 2025Option Discovery Using LLM-guided Semantic Hierarchical Reinforcement Learningcited by 4
  21. 2024Guiding Reinforcement Learning Using Uncertainty-Aware Large Language Modelscited by 3
  22. 2024LLM-Based Offline Learning for Embodied Agents via Consistency-Guided Reward Ensemblecited by 3
  23. 2025PCGRLLM: Large Language Model-Driven Reward Design for Procedural Content Generation Reinforcement Learningcited by 3
  24. 2025Heterogeneous Adversarial Play in Interactive Environmentscited by 3
  25. 2025Automated Reward Design for Gran Turismocited by 3
  26. 2025PROF: An LLM-based Reward Code Preference Optimization Framework for Offline Imitation Learningcited by 3
  27. 2024Training Fast Robot Policies with Slow Foundation Modelscited by 2
  28. 2025AED: Automatic Discovery of Effective and Diverse Vulnerabilities for Autonomous Driving Policy with Large Language Modelscited by 2
  29. 2025Leveraging LLMs for reward function design in reinforcement learning control taskscited by 2
  30. 2026Retrospective In-Context Learning for Temporal Credit Assignment with Large Language Modelscited by 2
  31. 2026EvoNav: Evolutionary Reward Function Design for Robot Navigation with Large Language Modelscited by 2
  32. 2026When LLM Reward Design Fails: Diagnostic-Driven Refinement for Sparse Structured RLcited by 2
  33. 2025An Automated Reinforcement Learning Reward Design Framework with Large Language Model for Cooperative Platoon Coordinationcited by 1
  34. 2025GuessBench: Sensemaking Multimodal Creativity in the Wildcited by 1
  35. 2025Agentic Episodic Controlcited by 1
  36. 2025VORTEX: Aligning Task Utility and Human Preferences through LLM-Guided Reward Shapingcited by 1
  37. 2025Guiding Exploration in Reinforcement Learning Through LLM-Augmented Observationscited by 1
  38. 2026MIMIC-Py: An Extensible Tool for Personality-Driven Automated Game Testing with Large Language Modelscited by 1

50 most cited of 61 All 61 (JSON)