# Reward Shaping via Diffusion Process in Reinforcement Learning

arXiv: https://arxiv.org/abs/2306.11885 · HTML: https://mlsanity.com/paper/2306.11885/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 0 of its 27 references that are in the map.
- First posted on arXiv 20 Jun 2023.

## Built on by (most cited first, 2 of 2)

- [SuS: Strategy-aware Surprise for Intrinsic Exploration](https://mlsanity.com/paper/2601.10349/) (2026)
- [When Does External Guidance Help LLM Reasoning? A Bias-Variance Theory of Guidance-Augmented GRPO](https://mlsanity.com/paper/2610.06861/) (2026)

## Abstract (arXiv)

Reinforcement Learning (RL) models have continually evolved to navigate the exploration - exploitation trade-off in uncertain Markov Decision Processes (MDPs). In this study, I leverage the principles of stochastic thermodynamics and system dynamics to explore reward shaping via diffusion processes. This provides an elegant framework as a way to think about exploration-exploitation trade-off. This article sheds light on relationships between information entropy, stochastic system dynamics, and their influences on entropy production. This exploration allows us to construct a dual-pronged framework that can be interpreted as either a maximum entropy program for deriving efficient policies or a modified cost optimization program accounting for informational costs and benefits. This work presents a novel perspective on the physical nature of information and its implications for online learning in MDPs, consequently providing a better understanding of information-oriented formulations in RL.