# Toward Computationally Efficient Inverse Reinforcement Learning via Reward Shaping

arXiv: https://arxiv.org/abs/2312.09983 · HTML: https://mlsanity.com/paper/2312.09983/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 9 of its 21 references that are in the map.
- First posted on arXiv 15 Dec 2023.

## Placed in

- Offline RL and imitation learning from data / [Inverse reinforcement learning](https://mlsanity.com/offlinerl/inverse-reinforcement-learning/)

## Builds on (most cited first, 9 of 9)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Maximum Entropy Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/1507.04888/) (2015)
- [Learning to Utilize Shaping Rewards: A New Approach of Reward Shaping](https://mlsanity.com/paper/2011.02669/) (2020)
- [Unpacking Reward Shaping: Understanding the Benefits of Reward Engineering on Sample Complexity](https://mlsanity.com/paper/2210.09579/) (2022)
- [Identifiability in inverse reinforcement learning](https://mlsanity.com/paper/2106.03498/) (2021)
- [Heuristic-Guided Reinforcement Learning](https://mlsanity.com/paper/2106.02757/) (2021)
- [Bridging RL Theory and Practice with the Effective Horizon](https://mlsanity.com/paper/2304.09853/) (2023)
- [Receding Horizon Inverse Reinforcement Learning](https://mlsanity.com/paper/2206.04477/) (2022)
- [Guaranteeing Control Requirements via Reward Shaping in Reinforcement Learning](https://mlsanity.com/paper/2311.10026/) (2023)

## Built on by (most cited first, 2 of 2)

- [EvIL: Evolution Strategies for Generalisable Imitation Learning](https://mlsanity.com/paper/2406.11905/) (2024)
- [Bootstrapped Reward Shaping](https://mlsanity.com/paper/2501.00989/) (2025)

## Abstract (arXiv)

Inverse reinforcement learning (IRL) is computationally challenging, with common approaches requiring the solution of multiple reinforcement learning (RL) sub-problems. This work motivates the use of potential-based reward shaping to reduce the computational burden of each RL sub-problem. This work serves as a proof-of-concept and we hope will inspire future developments towards computationally efficient IRL.