mlsanityData to 14 Oct 2026

RLeXplore: Accelerating Research in Intrinsically-Motivated Reinforcement Learning

arXiv 2405.19548 ↗First posted 29 May 2024cs.LGcited by 10

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Extrinsic rewards can effectively guide reinforcement learning (RL) agents in specific tasks. However, extrinsic rewards frequently fall short in complex environments due to the significant human effort needed for their design and annotation. This limitation underscores the necessity for intrinsic rewards, which offer auxiliary and dense signals and can enable agents to learn in an unsupervised manner. Although various intrinsic reward formulations have been proposed, their implementation and optimization details are insufficiently explored and lack standardization, thereby hindering research progress. To address this gap, we introduce RLeXplore, a unified, highly modularized, and plug-and-play framework offering reliable implementations of eight state-of-the-art intrinsic reward methods. Furthermore, we conduct an in-depth study that identifies critical implementation details and establishes well-justified standard practices in intrinsically-motivated RL. Our documentation, examples, and source code are available at https://github.com/RLE-Foundation/RLeXplore.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 37 of its 54 references that are in the map. First posted on arXiv 29 May 2024.

Builds on

37 of 54 references.

References · 37

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  5. 2018Exploration by Random Network Distillationcited by 993
  6. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  7. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  8. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  9. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  10. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 327
  11. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 309
  12. 2018Large-Scale Study of Curiosity-Driven Learningcited by 280
25 more
  1. 2019Self-Supervised Exploration via Disagreementcited by 265
  2. 2017Count-Based Exploration with Neural Density Modelscited by 257
  3. 2020Never Give Up: Learning Directed Exploration Strategiescited by 212
  4. 2020Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPOcited by 186
  5. 2015Incentivizing Exploration In Reinforcement Learning With Deep Predictive Modelscited by 154
  6. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 139
  7. 2020RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environmentscited by 135
  8. 2018Count-Based Exploration with the Successor Representationcited by 112
  9. 2018Episodic Curiosity through Reachabilitycited by 109
  10. 2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 108
  11. 2024Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learningcited by 88
  12. 2022BYOL-Explore: Exploration by Bootstrapped Predictioncited by 73
  13. 2022Exploration via Elliptical Episodic Bonusescited by 51
  14. 2017Count-Based Exploration in Feature Space for Reinforcement Learningcited by 49
  15. 2021On Bonus-Based Exploration Methods in the Arcade Learning Environmentcited by 37
  16. 2020Intrinsic Reward Driven Imitation Learning via Generative Modelcited by 35
  17. 2020BeBold: Exploration Beyond the Boundary of Explored Regionscited by 33
  18. 2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 28
  19. 2023Flipping Coins to Estimate Pseudocounts for Exploration in Reinforcement Learningcited by 28
  20. 2022General Intelligence Requires Rethinking Explorationcited by 20
  21. 2023A Study of Global and Episodic Bonuses for Exploration in Contextual MDPscited by 20
  22. 2022LECO: Learnable Episodic Count for Task-Specific Intrinsic Rewardcited by 11
  23. 2023Unlocking the Power of Representations in Long-term Novelty-based Explorationcited by 9
  24. 2023Improving Intrinsic Exploration by Creating Stationary Objectivescited by 7
  25. 2023RLLTE: Long-Term Evolution Project of Reinforcement Learningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 6; 2026: 3.

10 papers.

  1. 2025The impact of intrinsic rewards on exploration in Reinforcement Learningcited by 8
  2. 2025Masking Teacher and Reinforcing Student for Distilling Vision-Language Modelscited by 6
  3. 2025ARM-FM: Automated Reward Machines via Foundation Models for Compositional Reinforcement Learningcited by 5
  4. 2024Syllabus: Portable Curricula for Reinforcement Learning Agentscited by 2
  5. 2025PvP: Data-Efficient Humanoid Robot Learning with Proprioceptive-Privileged Contrastive Representationscited by 2
  6. 2025Deep Reinforcement Learning with Hybrid Intrinsic Reward Modelcited by 1
  7. 2025Mixture of Autoencoder Experts Guidance using Unlabeled and Incomplete Data for Exploration in Reinforcement Learningcited by 1
  8. 2026Online Reward-Punishment Learning from Fixed-Channel Perceptual Event Streams without Environment Rewardscited by 0
  9. 2026Information-Based Exploration via Random Features for Reinforcement Learningcited by 0
  10. 2026Reward Structure Shapes the Interaction Between Episodic Exploration and Neural Memory in Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards better dense rewards in Reinforcement Learning Applications
  2. 2025LLM-Driven Intrinsic Motivation for Sparse Reward Reinforcement Learning
  3. 2024Effective Reward Specification in Deep Reinforcement Learning
  4. 2024The Distributional Reward Critic Framework for Reinforcement Learning Under Perturbed Rewards
  5. 2023Intrinsic Motivation in Model-based Reinforcement Learning: A Brief Review
  6. 2025Reward Models in Deep Reinforcement Learning: A Survey
  7. 2024Potential-Based Intrinsic Motivation: Preserving Optimality With Complex, Non-Markovian Shaping Rewards
  8. 2025Minding Motivation: The Effect of Intrinsic Motivation on Agent Behaviors