mlsanityData to 7 Oct 2026

Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Prediction

arXiv 2301.10034 ↗First posted 21 Jan 2023cs.AIcited by 39

Not on the map. Why?

Abstract (arXiv)

We study the problem of learning goal-conditioned policies in Minecraft, a popular, widely accessible yet challenging open-ended environment for developing human-level multi-task agents. We first identify two main challenges of learning such policies: 1) the indistinguishability of tasks from the state distribution, due to the vast scene diversity, and 2) the non-stationary nature of environment dynamics caused by partial observability. To tackle the first challenge, we propose Goal-Sensitive Backbone (GSB) for the policy to encourage the emergence of goal-relevant visual state representations. To tackle the second challenge, the policy is further fueled by an adaptive horizon prediction module that helps alleviate the learning uncertainty brought by the non-stationary dynamics. Experiments on 20 Minecraft tasks show that our method significantly outperforms the best baseline so far; in many of them, we double the performance. Our ablation and exploratory studies then explain how our approach beat the counterparts and also unveil the surprising bonus of zero-shot generalization to new scenes (biomes). We hope our agent could help shed some light on learning goal-conditioned, multi-task agents in challenging, open-ended environments like Minecraft.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Builds on 42 of its 62 references that are in the map. First posted on arXiv 21 Jan 2023.

Builds on

42 of 62 references.

References · 42

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  5. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  6. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  7. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  8. 2022Visual Prompt Tuningcited by 1,674
  9. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  10. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  11. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  12. 2018DeepMind Control Suitecited by 935
30 more
  1. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  2. 2022A Generalist Agentcited by 802
  3. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
  4. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  5. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  6. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 338
  7. 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 268
  8. 2020First return, then explorecited by 256
  9. 2018BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learningcited by 238
  10. 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 210
  11. 2019Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutionscited by 182
  12. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 177
  13. 2019Self-Monitoring Navigation Agent via Auxiliary Progress Estimationcited by 173
  14. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  15. 2021BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environmentscited by 161
  16. 2019Goal-conditioned Imitation Learningcited by 159
  17. 2019Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actionscited by 124
  18. 2019The Regretful Agent: Heuristic-Aided Navigation through Progress Estimationcited by 104
  19. 2017Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learningcited by 97
  20. 2021Continual Backprop: Stochastic Gradient Descent with Persistent Randomnesscited by 79
  21. 2020Imitating Interactive Intelligencecited by 55
  22. 2019Obstacle Tower: A Generalization Challenge in Vision, Control, and Planningcited by 54
  23. 2016Trace Norm Regularised Deep Multi-Task Learningcited by 50
  24. 2021SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policiescited by 42
  25. 2021The MineRL 2020 Competition on Sample Efficient Reinforcement Learning using Human Priorscited by 40
  26. 2021JueWu-MC: Playing Minecraft with Sample-efficient Hierarchical Reinforcement Learningcited by 28
  27. 2022MineRL Diamond 2021 Competition: Overview, Results, and Lessons Learnedcited by 26
  28. 2019The MineRL 2019 Competition on Sample Efficient Reinforcement Learning using Human Priorscited by 25
  29. 2021SEIHAI: A Sample-efficient Hierarchical AI for the MineRL Competitioncited by 20
  30. 2020StarCraft II Build Order Optimization using Deep Reinforcement Learning and Monte-Carlo Tree Searchcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 15; 2024: 12; 2025: 9; 2026: 3.

39 papers.

Sort:

By area:

  1. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  2. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 396
  3. 2023An Embodied Generalist Agent in 3D Worldcited by 355
  4. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memorycited by 288
  5. 2023JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Modelscited by 170
  6. 2023STEVE-1: A Generative Model for Text-to-Behavior in Minecraftcited by 95
  7. 2024Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Taskscited by 83
  8. 2024RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Horizon Generationcited by 75
  9. 2023Octopus: Embodied Vision-Language Programmer from Environmental Feedbackcited by 73
  10. 2023MP5: A Multi-modal Open-ended Embodied System in Minecraft via Active Perceptioncited by 67
  11. 2023GROOT: Learning to Follow Instructions by Watching Gameplay Videoscited by 42
  12. 2024Vision-Language Models Provide Promptable Representations for Reinforcement Learningcited by 41
27 more
  1. 2023Skill Reinforcement Learning and Planning for Open-World Long-Horizon Taskscited by 35
  2. 2024OmniJARVIS: Unified Vision-Language-Action Tokenization Enables Open-World Instruction Following Agentscited by 30
  3. 2025JARVIS-VLA: Post-Training Large-Scale Vision Language Models to Play Visual Games with Keyboards and Mousecited by 27
  4. 2024Odyssey: Empowering Minecraft Agents with Open-World Skillscited by 26
  5. 2023Creative Agents: Empowering Agents with Imagination for Creative Taskscited by 24
  6. 2025Optimus-2: Multimodal Minecraft Agent with Goal-Observation-Action Conditioned Policycited by 24
  7. 2024ROCKET-1: Mastering Open-World Interaction with Visual-Temporal Context Promptingcited by 20
  8. 2024Do We Really Need a Complex Agent System? Distill Embodied Agent into a Single Modelcited by 19
  9. 2024MineLand: Simulating Large-Scale Multi-Agent Interactions with Limited Multimodal Senses and Physical Needscited by 16
  10. 2025NeSyC: A Neuro-symbolic Continual Learner For Complex Embodied Tasks In Open Domainscited by 15
  11. 2023Reinforcement Learning Friendly Vision-Language Model for Minecraftcited by 14
  12. 2025Position: Interactive Generative Video as Next-Generation Game Enginecited by 13
  13. 2023Active Reasoning in an Open-World Environmentcited by 8
  14. 2024MrSteve: Instruction-Following Agents in Minecraft with What-Where-When Memorycited by 8
  15. 2025Open-World Skill Discovery from Unsegmented Demonstrationscited by 7
  16. 2023Visual Encoders for Data-Efficient Imitation Learning in Modern Video Gamescited by 6
  17. 2023PDiT: Interleaving Perception and Decision-making Transformers for Deep Reinforcement Learningcited by 5
  18. 2024Read to Play (R2-Play): Decision Transformer with Multimodal Game Instructioncited by 4
  19. 2024Visual Grounding for Object-Level Generalization in Reinforcement Learningcited by 4
  20. 2025Deep RL Needs Deep Behavior Analysis: Exploring Implicit Planning by Model-Free Agents in Open-Ended Environmentscited by 4
  21. 2026Experience Transfer for Multimodal LLM Agents in Minecraft Gamecited by 4
  22. 2024Preference Goal Tuning: Post-Training as Latent Control for Frozen Policiescited by 3
  23. 2025RIG: Synergizing Reasoning and Imagination in End-to-End Generalist Policycited by 3
  24. 2025Scalable Multi-Task Reinforcement Learning for Generalizable Spatial Intelligence in Visuomotor Agentscited by 3
  25. 2025Goal-Guided Efficient Exploration via Large Language Model in Reinforcement Learningcited by 0
  26. 2026HAD: Heterogeneity-Aware Distillation for Lifelong Heterogeneous Learningcited by 0
  27. 2026WISE: A Long-Horizon Agent in Minecraft with Why-Which Reasoningcited by 0