mlsanityData to 14 Oct 2026

BeBold: Exploration Beyond the Boundary of Explored Regions

arXiv 2012.08621 ↗First posted 15 Dec 2020cs.LG cs.AI stat.MLcited by 33

Not on the map. Why?

Abstract (arXiv)

Efficient exploration under sparse rewards remains a key challenge in deep reinforcement learning. To guide exploration, previous work makes extensive use of intrinsic reward (IR). There are many heuristics for IR, including visitation counts, curiosity, and state-difference. In this paper, we analyze the pros and cons of each method and propose the regulated difference of inverse visitation counts as a simple but effective criterion for IR. The criterion helps the agent explore Beyond the Boundary of explored regions and mitigates common issues in count-based methods, such as short-sightedness and detachment. The resulting method, BeBold, solves the 12 most challenging procedurally-generated tasks in MiniGrid with just 120M environment steps, without any curriculum learning. In comparison, the previous SoTA only solves 50% of the tasks. BeBold also achieves SoTA on multiple tasks in NetHack, a popular rogue-like game that contains more challenging procedurally-generated environments.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). First posted on arXiv 15 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 10; 2022: 5; 2023: 10; 2024: 2; 2025: 3; 2026: 3.

33 papers.

Sort:

By area:

  1. 2021MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Researchcited by 97
  2. 2023XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAXcited by 55
  3. 2021MADE: Exploration via Maximizing Deviation from Explored Regionscited by 40
  4. 2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 28
  5. 2023The SocialAI School: Insights from Developmental Psychology Towards Artificial Socio-Cultural Agentscited by 24
  6. 2021Dynamic Bottleneck for Robust Self-Supervised Explorationcited by 21
  7. 2023TiZero: Mastering Multi-Agent Football with Curriculum Learning and Self-Playcited by 20
  8. 2021TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrationscited by 18
  9. 2024NAVIX: Scaling MiniGrid Environments with JAXcited by 18
  10. 2021C-Planning: An Automatic Curriculum for Learning Goal-Reaching Taskscited by 17
  11. 2021Generalization in Text-based Games via Hierarchical Reinforcement Learningcited by 11
  12. 2023minimax: Efficient Baselines for Autocurricula in JAXcited by 10
21 more
  1. 2024RLeXplore: Accelerating Research in Intrinsically-Motivated Reinforcement Learningcited by 10
  2. 2021Focus on Impact: Indoor Exploration with Intrinsic Motivationcited by 9
  3. 2021Don't Do What Doesn't Matter: Intrinsic Motivation with Action Usefulnesscited by 7
  4. 2022Generative Personas That Behave and Experience Like Humanscited by 7
  5. 2021Explore and Control with Adversarial Surprisecited by 6
  6. 2022Play with Emotion: Affect-Driven Reinforcement Learningcited by 6
  7. 2023Approximate information state based convergence analysis of recurrent Q-learningcited by 5
  8. 2022Maze Learning using a Hyperdimensional Predictive Processing Cognitive Architecturecited by 4
  9. 2023Learning to Solve Tasks with Exploring Prior Behaviourscited by 4
  10. 2025β-DQN: Improving Deep Q-Learning By Evolving the Behaviorcited by 3
  11. 2026AI Finds A Waycited by 3
  12. 2021Co-Imitation Learning without Expert Demonstrationcited by 2
  13. 2022BIMRL: Brain Inspired Meta Reinforcement Learningcited by 2
  14. 2023A Neuro-mimetic Realization of the Common Model of Cognition via Hebbian Learning and Free Energy Minimizationcited by 2
  15. 2023Learning Curricula in Open-Ended Worldscited by 2
  16. 2023OpenRL: A Unified Reinforcement Learning Frameworkcited by 2
  17. 2023Using Offline Data to Speed Up Reinforcement Learning in Procedurally Generated Environmentscited by 1
  18. 2025Is Exploration or Optimization the Problem for Deep Reinforcement Learning?cited by 1
  19. 2025Robust Agents in Open-Ended Worldscited by 1
  20. 2026Beyond Single-Model Optimization: Preserving Plasticity in Continual Reinforcement Learningcited by 1
  21. 2026Explore Beyond the Boundary Using Entropic Informationcited by 0