mlsanityData to 14 Oct 2026

Reward Inflation: A Healthy Stimulus for Reinforcement Learning

arXiv 2610.02545 ↗First posted 1 Oct 2026cs.LGcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Reward serves as the primary learning signal in reinforcement learning (RL). However, while reward magnitudes are typically held fixed throughout training, their temporal modulation remains underexplored. In this paper, we propose reward inflation, a gradual scaling of rewards over the course of training, and show that it can act as a healthy stimulus for RL. Theoretically, reward inflation induces an implicit recency weighting that upweights recent transitions during policy updates, enabling faster adaptation. We further show that, by sustaining gradient signals as the policy saturates, reward inflation suppresses the emergence of dormant neurons and helps preserve plasticity. Empirical results on ALE games and MuJoCo tasks corroborate these findings, showing that an appropriate level of reward inflation benefits a broad range of tasks. Finally, we introduce Fed, an adaptive variant that adjusts the inflation level on the fly, and find that it often improves upon fixed inflation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 40 of its 56 references that are in the map. First posted on arXiv 1 Oct 2026.

Builds on

40 of 56 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Reward Prediction Error Prioritisation in Experience Replay: The RPE-PER Method30 Jan
  2. 2025From Sparse to Dense: Toddler-inspired Reward Transition in Goal-Oriented Reinforcement Learning29 Jan
  3. 2024Curriculum Reinforcement Learning for Complex Reward Functions22 Oct
  4. 2024Highly Efficient Self-Adaptive Reward Shaping for Reinforcement Learning6 Aug
  5. 2024Normalization and effective learning rates in reinforcement learning1 Jul
  6. 2024Disentangling the Causes of Plasticity Loss in Neural Networks29 Feb

Other references · 34

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  3. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  4. 2015Prioritized Experience Replaycited by 1,099
  5. 2017Deep Reinforcement Learning that Matterscited by 754
  6. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 303
  7. 2022The Primacy Bias in Deep Reinforcement Learningcited by 247
  8. 2021Reinforcement learning based recommender systems: A surveycited by 188
  9. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 176
  10. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 174
  11. 2023Understanding plasticity in neural networkscited by 174
  12. 2018Multi-task Deep Reinforcement Learning with PopArtcited by 141
22 more
  1. 2020Learning to Utilize Shaping Rewards: A New Approach of Reward Shapingcited by 114
  2. 2016Learning values across many orders of magnitudecited by 85
  3. 2019A survey on intrinsic motivation in reinforcement learningcited by 81
  4. 2019Experience Replay Optimizationcited by 48
  5. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecyclecited by 35
  6. 2019Reward Shaping via Meta-Learningcited by 32
  7. 2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 28
  8. 2021Return-based Scaling: Yet Another Normalisation Trick for Deep RLcited by 23
  9. 2022Prioritizing Samples in Reinforcement Learning with Reducible Losscited by 22
  10. 2022Actor Prioritized Experience Replaycited by 20
  11. 2019Boosting Soft Actor-Critic: Emphasizing Recent Experience without Forgetting the Pastcited by 18
  12. 2024A Review of Reinforcement Learning in Financial Applicationscited by 13
  13. 2023Automatic Intrinsic Reward Shaping for Exploration in Deep Reinforcement Learningcited by 12
  14. 2020The effect of Target Normalization and Momentum on Dying ReLUcited by 6
  15. 2022A survey on recently proposed activation functions for Deep Learningcited by 6
  16. 2024Maxwell's Demon at Work: Efficient Pruning by Leveraging Saturation of Neuronscited by 6
  17. 2025The Resurrection of the ReLUcited by 6
  18. 2024Demystifying the Recency Heuristic in Temporal-Difference Learningcited by 4
  19. 2021Adapting to Reward Progressivity via Spectral Reinforcement Learningcited by 3
  20. 2023Automaton-Guided Curriculum Generation for Reinforcement Learning Agentscited by 2
  21. 2022Achieving Goals using Reward Shaping and Curriculum Learningcited by 1
  22. 2023Neural Collage Transfer: Artistic Reconstruction via Material Manipulationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Zero-Incentive Dynamics: a look at reward sparsity through the lens of unrewarded subgoals
  2. 2026A Unified Framework for Dynamic Reward Shaping in Reinforcement Learning
  3. 2026Fine-Tuning on Self-Generated and Reward-Weighted Data: Learning Dynamics, Convergence Rates, and Benefits of Off-Policyness
  4. 2025Reducing Reward Dependence in RL Through Adaptive Confidence Discounting
  5. 2026When Sparse Reward Meets Dense Distillation: Training Dynamics of On-Policy Distillation
  6. 2026Reward-Conditioned Reinforcement Learning
  7. 2026Reward Structure Shapes the Interaction Between Episodic Exploration and Neural Memory in Reinforcement Learning
  8. 2026Tackling Length Inflation Without Trade-offs: Group Relative Reward Rescaling for Reinforcement Learning