Plan-Based Relaxed Reward Shaping for Goal-Directed Tasks
Not on the map. Why?
Abstract (arXiv)
In high-dimensional state spaces, the usefulness of Reinforcement Learning (RL) is limited by the problem of exploration. This issue has been addressed using potential-based reward shaping (PB-RS) previously. In the present work, we introduce Final-Volume-Preserving Reward Shaping (FV-RS). FV-RS relaxes the strict optimality guarantees of PB-RS to a guarantee of preserved long-term behavior. Being less restrictive, FV-RS allows for reward shaping functions that are even better suited for improving the sample efficiency of RL algorithms. In particular, we consider settings in which the agent has access to an approximate plan. Here, we use examples of simulated robotic manipulation tasks to demonstrate that plan-based FV-RS can indeed significantly improve the sample efficiency of RL over plan-based PB-RS.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 5 of its 34 references that are in the map. First posted on arXiv 14 Jul 2021.
Builds on
5 of 34 references.
References · 5
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2015Continuous control with deep reinforcement learningcited by 3,407
- 2017Hindsight Experience Replaycited by 1,136
- 2016TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systemscited by 664
- 2018Learning Montezuma's Revenge from a Single Demonstrationcited by 80
Built on by
Citing papers per year: 2021: 1; 2022: 2; 2023: 1; 2024: 1; 2025: 0; 2026: 1.
6 papers.
- 2022Calculus on MDPs: Potential Shaping as a Gradientcited by 4
- 2024Reward Guidance for Reinforcement Learning Tasks Based on Large Language Models: The LMGT Frameworkcited by 4
- 2021Learning to Execute: Efficient Learning of Universal Plan-Conditioned Policies in Roboticscited by 2
- 2022D-Shape: Demonstration-Shaped Reinforcement Learning via Goal Conditioningcited by 2
- 2023Exploiting Multiple Abstractions in Episodic RL via Reward Shapingcited by 2
- 2026Path-conditioned Reinforcement Learning-based Local Planning for Long-Range Navigationcited by 0
- 2026Path-conditioned Reinforcement Learning-based Local Planning for Long-Range Navigation14 Mar 2026
- 2024Reward Guidance for Reinforcement Learning Tasks Based on Large Language Models: The LMGT Framework7 Sep 2024
- 2023Exploiting Multiple Abstractions in Episodic RL via Reward Shaping28 Feb 2023
- 2022D-Shape: Demonstration-Shaped Reinforcement Learning via Goal Conditioning26 Oct 2022
- 2022Calculus on MDPs: Potential Shaping as a Gradient20 Aug 2022
- 2021Learning to Execute: Efficient Learning of Universal Plan-Conditioned Policies in Robotics15 Nov 2021