# Persistent Rule-based Interactive Reinforcement Learning

arXiv: https://arxiv.org/abs/2102.02441 · HTML: https://mlsanity.com/paper/2102.02441/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 69 references that are in the map.
- First posted on arXiv 4 Feb 2021.

## Builds on (most cited first, 9 of 9)

- [Towards Practical Multi-Object Manipulation using Relational Reinforcement Learning](https://mlsanity.com/paper/1912.11032/) (2019)
- [A Conceptual Framework for Externally-influenced Agents: An Assisted Reinforcement Learning Review](https://mlsanity.com/paper/2007.01544/) (2020)
- [Deep Reinforcement Learning with Interactive Feedback in a Human-Robot Environment](https://mlsanity.com/paper/2007.03363/) (2020)
- [Multi-modal Feedback for Affordance-driven Interactive Reinforcement Learning](https://mlsanity.com/paper/1807.09991/) (2018)
- [Improving interactive reinforcement learning: What makes a good teacher?](https://mlsanity.com/paper/1904.06879/) (2019)
- [Human Engagement Providing Evaluative and Informative Advice for Interactive Reinforcement Learning](https://mlsanity.com/paper/2009.09575/) (2020)
- [iCub: Learning Emotion Expressions using Human Reward](https://mlsanity.com/paper/2003.13483/) (2020)
- [Unmanned Aerial Vehicle Control Through Domain-based Automatic Speech Recognition](https://mlsanity.com/paper/2009.04215/) (2020)
- [Multiple decision trees](https://mlsanity.com/paper/1304.2363/) (2013)

## Built on by (most cited first, 5 of 5)

- [Multi-trainer Interactive Reinforcement Learning System](https://mlsanity.com/paper/2210.08050/) (2022)
- [Broad-persistent Advice for Interactive Reinforcement Learning Scenarios](https://mlsanity.com/paper/2210.05187/) (2022)
- [Reinforcement Learning for UAV control with Policy and Reward Shaping](https://mlsanity.com/paper/2212.03828/) (2022)
- [Agent-Aware Training for Agent-Agnostic Action Advising in Deep Reinforcement Learning](https://mlsanity.com/paper/2311.16807/) (2023)
- [Human-Allied Relational Reinforcement Learning](https://mlsanity.com/paper/2510.16188/) (2025)

## Abstract (arXiv)

Interactive reinforcement learning has allowed speeding up the learning process in autonomous agents by including a human trainer providing extra information to the agent in real-time. Current interactive reinforcement learning research has been limited to real-time interactions that offer relevant user advice to the current state only. Additionally, the information provided by each interaction is not retained and instead discarded by the agent after a single-use. In this work, we propose a persistent rule-based interactive reinforcement learning approach, i.e., a method for retaining and reusing provided knowledge, allowing trainers to give general advice relevant to more than just the current state. Our experimental results show persistent advice substantially improves the performance of the agent while reducing the number of interactions required for the trainer. Moreover, rule-based advice shows similar performance impact as state-based advice, but with a substantially reduced interaction count.