# A Fully Controllable Agent in the Path Planning using Goal-Conditioned Reinforcement Learning

arXiv: https://arxiv.org/abs/2205.09967 · HTML: https://mlsanity.com/paper/2205.09967/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 63 references that are in the map.
- First posted on arXiv 20 May 2022.

## Builds on (most cited first, 25 of 25)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Exploration by Random Network Distillation](https://mlsanity.com/paper/1810.12894/) (2018)
- [Data-Efficient Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1805.08296/) (2018)
- [Visual Reinforcement Learning with Imagined Goals](https://mlsanity.com/paper/1807.04742/) (2018)
- [Search on the Replay Buffer: Bridging Planning and Reinforcement Learning](https://mlsanity.com/paper/1906.05253/) (2019)
- [Goal-Conditioned Reinforcement Learning with Imagined Subgoals](https://mlsanity.com/paper/2107.00541/) (2021)
- [C-Learning: Learning to Achieve Goals via Recursive Classification](https://mlsanity.com/paper/2011.08909/) (2020)
- [Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RL](https://mlsanity.com/paper/2202.04478/) (2022)
- [World Model as a Graph: Learning Latent Landmarks for Planning](https://mlsanity.com/paper/2011.12491/) (2020)
- [Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2110.13625/) (2021)
- [Maximum Entropy-Regularized Multi-Goal Reinforcement Learning](https://mlsanity.com/paper/1905.08786/) (2019)
- [Mobile Robot Path Planning in Dynamic Environments through Globally Guided Reinforcement Learning](https://mlsanity.com/paper/2005.05420/) (2020)
- [Deep Reinforcement Learning based Automatic Exploration for Navigation in Unknown Environment](https://mlsanity.com/paper/2007.11808/) (2020)
- [Self-Imitation Learning by Planning](https://mlsanity.com/paper/2103.13834/) (2021)
- [Hierarchical Reinforcement Learning with Timed Subgoals](https://mlsanity.com/paper/2112.03100/) (2021)
- [Weakly-Supervised Reinforcement Learning for Controllable Behavior](https://mlsanity.com/paper/2004.02860/) (2020)
- [Multi-UAV Path Planning for Wireless Data Harvesting with Deep Reinforcement Learning](https://mlsanity.com/paper/2010.12461/) (2020)
- [Subgoal-based Reward Shaping to Improve Efficiency in Reinforcement Learning](https://mlsanity.com/paper/2104.06411/) (2021)
- [Hierarchies of Planning and Reinforcement Learning for Robot Navigation](https://mlsanity.com/paper/2109.11178/) (2021)
- [Hierarchical Reinforcement Learning Method for Autonomous Vehicle Behavior Planning](https://mlsanity.com/paper/1911.03799/) (2019)
- [Learning user-defined sub-goals using memory editing in reinforcement learning](https://mlsanity.com/paper/2205.00399/) (2022)

## Abstract (arXiv)

The aim of path planning is to reach the goal from starting point by searching for the route of an agent. In the path planning, the routes may vary depending on the number of variables such that it is important for the agent to reach various goals. Numerous studies, however, have dealt with a single goal that is predefined by the user. In the present study, I propose a novel reinforcement learning framework for a fully controllable agent in the path planning. To do this, I propose a bi-directional memory editing to obtain various bi-directional trajectories of the agent, in which the behavior of the agent and sub-goals are trained on the goal-conditioned RL. As for moving the agent in various directions, I utilize the sub-goals dedicated network, separated from a policy network. Lastly, I present the reward shaping to shorten the number of steps for the agent to reach the goal. In the experimental result, the agent was able to reach the various goals that have never been visited by the agent in the training. We confirmed that the agent could perform difficult missions such as a round trip and the agent used the shorter route with the reward shaping.