# Learning user-defined sub-goals using memory editing in reinforcement learning

arXiv: https://arxiv.org/abs/2205.00399 · HTML: https://mlsanity.com/paper/2205.00399/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 34 references that are in the map.
- First posted on arXiv 1 May 2022.

## Builds on (most cited first, 14 of 14)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Exploration by Random Network Distillation](https://mlsanity.com/paper/1810.12894/) (2018)
- [Unifying Count-Based Exploration and Intrinsic Motivation](https://mlsanity.com/paper/1606.01868/) (2016)
- [Count-Based Exploration with Neural Density Models](https://mlsanity.com/paper/1703.01310/) (2017)
- [Sample Efficient Actor-Critic with Experience Replay](https://mlsanity.com/paper/1611.01224/) (2016)
- [Count-Based Exploration with the Successor Representation](https://mlsanity.com/paper/1807.11622/) (2018)
- [Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learning](https://mlsanity.com/paper/2007.02832/) (2020)
- [Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2110.13625/) (2021)
- [DORA The Explorer: Directed Outreaching Reinforcement Action-Selection](https://mlsanity.com/paper/1804.04012/) (2018)
- [Many-Goals Reinforcement Learning](https://mlsanity.com/paper/1806.09605/) (2018)
- [Weakly-Supervised Reinforcement Learning for Controllable Behavior](https://mlsanity.com/paper/2004.02860/) (2020)
- [Subgoal-based Reward Shaping to Improve Efficiency in Reinforcement Learning](https://mlsanity.com/paper/2104.06411/) (2021)

## Built on by (most cited first, 1 of 1)

- [A Fully Controllable Agent in the Path Planning using Goal-Conditioned Reinforcement Learning](https://mlsanity.com/paper/2205.09967/) (2022)

## Abstract (arXiv)

The aim of reinforcement learning (RL) is to allow the agent to achieve the final goal. Most RL studies have focused on improving the efficiency of learning to achieve the final goal faster. However, the RL model is very difficult to modify an intermediate route in the process of reaching the final goal. That is, the agent cannot be under control to achieve other sub-goals in the existing studies. If the agent can go through the sub-goals on the way to the destination, the RL can be applied and studied in various fields. In this study, I propose a methodology to achieve the user-defined sub-goals as well as the final goal using memory editing. The memory editing is performed to generate various sub-goals and give an additional reward to the agent. In addition, the sub-goals are separately learned from the final goal. I set two simple environments and various scenarios in the test environments. As a result, the agent almost successfully passed the sub-goals as well as the final goal under control. Moreover, the agent was able to be induced to visit the novel state indirectly in the environments. I expect that this methodology can be used in the fields that need to control the agent in a variety of scenarios.