# Reward Propagation Using Graph Convolutional Networks

arXiv: https://arxiv.org/abs/2010.02474 · HTML: https://mlsanity.com/paper/2010.02474/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Oct 2020.

## Built on by (most cited first, 15 of 15)

- [Deep Laplacian-based Options for Temporally-Extended Exploration](https://mlsanity.com/paper/2301.11181/) (2023)
- [Off-Policy Reinforcement Learning with Delayed Rewards](https://mlsanity.com/paper/2106.11854/) (2021)
- [Flexible Option Learning](https://mlsanity.com/paper/2112.03097/) (2021)
- [Neural Algorithmic Reasoners are Implicit Planners](https://mlsanity.com/paper/2110.05442/) (2021)
- [Provably Efficient Offline Reinforcement Learning with Trajectory-Wise Reward](https://mlsanity.com/paper/2206.06426/) (2022)
- [Snowflake: Scaling GNNs to High-Dimensional Continuous Control via Parameter Freezing](https://mlsanity.com/paper/2103.01009/) (2021)
- [Reward Shaping Using Convolutional Neural Network](https://mlsanity.com/paper/2210.16956/) (2022)
- [Incorporating Spatial Information into Goal-Conditioned Hierarchical Reinforcement Learning via Graph Representations](https://mlsanity.com/paper/2511.10872/) (2025)
- [Off-Beat Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2205.13718/) (2022)
- [On Addressing the Limitations of Graph Neural Networks](https://mlsanity.com/paper/2306.12640/) (2023)
- [Shaping Advice in Deep Reinforcement Learning](https://mlsanity.com/paper/2202.09489/) (2022)
- [Dealing with Sparse Rewards Using Graph Neural Networks](https://mlsanity.com/paper/2203.13424/) (2022)
- [Transductive Reward Inference on Graph](https://mlsanity.com/paper/2402.03661/) (2024)
- [The Mechanism Matters: When Knowledge Graphs Help Reinforcement Learning](https://mlsanity.com/paper/2607.19616/) (2026)
- [From Connectivity to Rewards: Dense Reward Learning with Directed State Graphs](https://mlsanity.com/paper/2609.10781/) (2026)

## Abstract (arXiv)

Potential-based reward shaping provides an approach for designing good reward functions, with the purpose of speeding up learning. However, automatically finding potential functions for complex environments is a difficult problem (in fact, of the same difficulty as learning a value function from scratch). We propose a new framework for learning potential functions by leveraging ideas from graph representation learning. Our approach relies on Graph Convolutional Networks which we use as a key ingredient in combination with the probabilistic inference view of reinforcement learning. More precisely, we leverage Graph Convolutional Networks to perform message passing from rewarding states. The propagated messages can then be used as potential functions for reward shaping to accelerate learning. We verify empirically that our approach can achieve considerable improvements in both small and high-dimensional control problems.