mlsanityData to 7 Oct 2026

Automatic Intrinsic Reward Shaping for Exploration in Deep Reinforcement Learning

arXiv 2301.10886 ↗First posted 26 Jan 2023cs.LG cs.AIcited by 11

Not on the map. Why?

Abstract (arXiv)

We present AIRS: Automatic Intrinsic Reward Shaping that intelligently and adaptively provides high-quality intrinsic rewards to enhance exploration in reinforcement learning (RL). More specifically, AIRS selects shaping function from a predefined set based on the estimated task return in real-time, providing reliable exploration incentives and alleviating the biased objective problem. Moreover, we develop an intrinsic reward toolkit to provide efficient and reliable implementations of diverse intrinsic reward approaches. We test AIRS on various tasks of MiniGrid, Procgen, and DeepMind Control Suite. Extensive simulation demonstrates that AIRS can outperform the benchmarking schemes and achieve superior performance with simple architecture.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 35 of its 53 references that are in the map. First posted on arXiv 26 Jan 2023.

Builds on

35 of 53 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2013Auto-Encoding Variational Bayescited by 3,980
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  6. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  7. 2018Exploration by Random Network Distillationcited by 966
  8. 2018DeepMind Control Suitecited by 935
  9. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  10. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  11. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  12. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
23 more
  1. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  2. 2017A Tutorial on Thompson Samplingcited by 420
  3. 2018Quantifying Generalization in Reinforcement Learningcited by 345
  4. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 307
  5. 2018Large-Scale Study of Curiosity-Driven Learningcited by 277
  6. 2017Count-Based Exploration with Neural Density Modelscited by 255
  7. 2020Never Give Up: Learning Directed Exploration Strategiescited by 210
  8. 2018An Introduction to Deep Reinforcement Learningcited by 201
  9. 2015Action-Conditional Video Prediction using Deep Networks in Atari Gamescited by 171
  10. 2015Incentivizing Exploration In Reinforcement Learning With Deep Predictive Modelscited by 154
  11. 2020RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environmentscited by 135
  12. 2018Count-Based Exploration with the Successor Representationcited by 112
  13. 2018Episodic Curiosity through Reachabilitycited by 109
  14. 2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 107
  15. 2018On Learning Intrinsic Rewards for Policy Gradient Methodscited by 100
  16. 2021Decoupling Value and Policy for Generalization in Reinforcement Learningcited by 81
  17. 2017Count-Based Exploration in Feature Space for Reinforcement Learningcited by 48
  18. 2020Intrinsic Reward Driven Imitation Learning via Generative Modelcited by 35
  19. 2018Learning to Play with Intrinsically-Motivated Self-Aware Agentscited by 34
  20. 2022R'enyi State Entropy for Exploration Acceleration in Reinforcement Learningcited by 18
  21. 2022LECO: Learnable Episodic Count for Task-Specific Intrinsic Rewardcited by 11
  22. 2022Rewarding Episodic Visitation Discrepancy for Exploration in Reinforcement Learningcited by 7
  23. 2022Tackling Visual Control via Multi-View Exploration Maximizationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 7; 2026: 1.

11 papers.

  1. 2025LAMARL: LLM-Aided Multi-Agent Reinforcement Learning for Cooperative Policy Generationcited by 13
  2. 2024PEAC: Unsupervised Pre-training for Cross-Embodiment Reinforcement Learningcited by 9
  3. 2025Automatic Reward Shaping from Confounded Offline Datacited by 6
  4. 2024Individual Contributions as Intrinsic Exploration Scaffolds for Multi-agent Reinforcement Learningcited by 4
  5. 2024Transformable Gaussian Reward Function for Socially-Aware Navigation with Deep Reinforcement Learningcited by 3
  6. 2025Action-Dependent Optimality-Preserving Reward Shapingcited by 2
  7. 2025Adaptive Data Exploitation in Deep Reinforcement Learningcited by 1
  8. 2025ULTHO: Ultra-Lightweight yet Efficient Hyperparameter Optimization in Deep Reinforcement Learningcited by 1
  9. 2025CAE: Repurposing the Critic as an Explorer in Deep Reinforcement Learningcited by 1
  10. 2025Topology Enhanced MARL for Multi-Agent Cooperative Decision-Making of CAVscited by 1
  11. 2026Adaptive Correlation-Weighted Intrinsic Rewards for Reinforcement Learningcited by 0