mlsanityData to 7 Oct 2026

Potential-Based Reward Shaping For Intrinsic Motivation

arXiv 2402.07411 ↗First posted 12 Feb 2024cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Recently there has been a proliferation of intrinsic motivation (IM) reward-shaping methods to learn in complex and sparse-reward environments. These methods can often inadvertently change the set of optimal policies in an environment, leading to suboptimal behavior. Previous work on mitigating the risks of reward shaping, particularly through potential-based reward shaping (PBRS), has not been applicable to many IM methods, as they are often complex, trainable functions themselves, and therefore dependent on a wider set of variables than the traditional reward functions that PBRS was developed for. We present an extension to PBRS that we prove preserves the set of optimal policies under a more general set of functions than has been previously proven. We also present em Potential-Based Intrinsic Motivation (PBIM), a method for converting IM rewards into a potential-based form that is useable without altering the set of optimal policies. Testing in the MiniGrid DoorKey and Cliff Walking environments, we demonstrate that PBIM successfully prevents the agent from converging to a suboptimal policy and can speed up training.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 10 of its 30 references that are in the map. First posted on arXiv 12 Feb 2024.

Builds on

10 of 30 references.

References · 10

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  3. 2018Exploration by Random Network Distillationcited by 966
  4. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  5. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  6. 2018Large-Scale Study of Curiosity-Driven Learningcited by 277
  7. 2017Count-Based Exploration with Neural Density Modelscited by 255
  8. 2015Variational Information Maximisation for Intrinsically Motivated Reinforcement Learningcited by 142
  9. 2020RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environmentscited by 135
  10. 2019Using Natural Language for Reward Shaping in Reinforcement Learningcited by 114

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 4; 2026: 5.

12 papers.

  1. 2025On the Importance of Reward Design in Reinforcement Learning-based Dynamic Algorithm Configuration: A Case Study on OneMax with (1+(λ,λ))-GAcited by 5
  2. 2024BAMDP Shaping: a Unified Framework for Intrinsic Motivation and Reward Shapingcited by 4
  3. 2026The Dark Room in the Reward Channel: Dense Prediction Rewards Collapse GRPO-Trained LLM Agents -- and The Channel, Not the Content, Decides What Workscited by 3
  4. 2024Potential-Based Intrinsic Motivation: Preserving Optimality With Complex, Non-Markovian Shaping Rewardscited by 2
  5. 2025Action-Dependent Optimality-Preserving Reward Shapingcited by 2
  6. 2024On the Sample Efficiency of Abstractions and Potential-Based Reward Shaping in Reinforcement Learningcited by 1
  7. 2025Deep Reinforcement Learning for Dynamic Algorithm Configuration: A Case Study on Optimizing OneMax with the (1+(λ,λ))-GAcited by 1
  8. 2026LLM-Enhanced Reinforcement Learning for Time Series Anomaly Detectioncited by 1
  9. 2025Minding Motivation: The Effect of Intrinsic Motivation on Agent Behaviorscited by 0
  10. 2026medR: Reward Engineering for Clinical Offline Reinforcement Learning via Tri-Drive Potential Functionscited by 0
  11. 2026Visual Set Program Synthesizercited by 0
  12. 2026MAGIC: Multi-Step Advantage-Gated Causal Influence for Multi-agent Reinforcement Learningcited by 0