mlsanityData to 14 Oct 2026

Exploration by Random Reward Perturbation

arXiv 2506.08737 ↗First posted 10 Jun 2025cs.LG cs.AIcited by 1

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

We introduce Random Reward Perturbation (RRP), a novel exploration strategy for reinforcement learning (RL). Our theoretical analyses demonstrate that adding zero-mean noise to environmental rewards effectively enhances policy diversity during training, thereby expanding the range of exploration. RRP is fully compatible with the action-perturbation-based exploration strategies, such as ε-greedy, stochastic policies, and entropy regularization, providing additive improvements to exploration effects. It is general, lightweight, and can be integrated into existing RL algorithms with minimal implementation effort and negligible computational overhead. RRP establishes a theoretical connection between reward shaping and noise-driven exploration, highlighting their complementary potential. Experiments show that RRP significantly boosts the performance of Proximal Policy Optimization and Soft Actor-Critic, achieving higher sample efficiency and escaping local optima across various tasks, under both sparse and dense reward scenarios.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 40 of its 65 references that are in the map. First posted on arXiv 10 Jun 2025.

Builds on

40 of 65 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Exploration by Random Distribution Distillation16 May
  2. 2024BAMDP Shaping: a Unified Framework for Intrinsic Motivation and Reward Shaping9 Sep
  3. 2024Highly Efficient Self-Adaptive Reward Shaping for Reinforcement Learning6 Aug
  4. 2024Comprehensive Overview of Reward Engineering and Shaping in Advancing Reinforcement Learning Applications22 Jul
  5. 2024Exploration and Anti-Exploration with Distributional Random Network Distillation18 Jan

Other references · 35

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015Continuous control with deep reinforcement learningcited by 3,407
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  5. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  6. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  7. 2018Exploration by Random Network Distillationcited by 993
  8. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  9. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
  10. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  11. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 309
  12. 2018Large-Scale Study of Curiosity-Driven Learningcited by 280
23 more
  1. 2017Noisy Networks for Explorationcited by 266
  2. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 260
  3. 2017Count-Based Exploration with Neural Density Modelscited by 257
  4. 2018Randomized Prior Functions for Deep Reinforcement Learningcited by 219
  5. 2020Never Give Up: Learning Directed Exploration Strategiescited by 212
  6. 2017Bridging the Gap Between Value and Policy Based Reinforcement Learningcited by 182
  7. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 166
  8. 2017Parameter Space Noise for Explorationcited by 161
  9. 2020RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environmentscited by 135
  10. 2018Count-Based Exploration with the Successor Representationcited by 112
  11. 2018Reinforcement Learning with Perturbed Rewardscited by 79
  12. 2017Boltzmann Exploration Done Rightcited by 69
  13. 2018Diversity-Driven Exploration Strategy for Deep Reinforcement Learningcited by 65
  14. 2017Count-Based Exploration in Feature Space for Reinforcement Learningcited by 49
  15. 2021Randomized Exploration for Reinforcement Learning with General Value Function Approximationcited by 49
  16. 2019Perturbed-History Exploration in Stochastic Linear Banditscited by 29
  17. 2023Flipping Coins to Estimate Pseudocounts for Exploration in Reinforcement Learningcited by 28
  18. 2018DORA The Explorer: Directed Outreaching Reinforcement Action-Selectioncited by 27
  19. 2021How to Stay Curious while Avoiding Noisy TVs using Aleatoric Uncertainty Estimationcited by 27
  20. 2024Randomized Exploration in Cooperative Multi-Agent Reinforcement Learningcited by 17
  21. 2019Perturbed-History Exploration in Stochastic Multi-Armed Banditscited by 16
  22. 2024Monte Carlo Tree Search with Boltzmann Explorationcited by 12
  23. 2021Learning to Shape Rewards using a Game of Two Partnerscited by 9

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Exploration vs Exploitation: Rethinking RLVR through Clipping, Entropy, and Spurious Rewardcited by 27

Similar, not cited

Close in wording, no citation either way

  1. 2025Learning to Explore in Diverse Reward Settings via Temporal-Difference-Error Maximization
  2. 2020Exploration by Maximizing R'enyi Entropy for Reward-Free RL Framework
  3. 2026Emergence of Exploration in Policy Gradient Reinforcement Learning via Retrying
  4. 2024Beyond Optimism: Exploration With Partially Observable Rewards
  5. 2024Deterministic Exploration via Stationary Bellman Error Maximization
  6. 2024The Distributional Reward Critic Framework for Reinforcement Learning Under Perturbed Rewards
  7. 2024Random Latent Exploration for Deep Reinforcement Learning
  8. 2024MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximization