# Multi-trainer Interactive Reinforcement Learning System

arXiv: https://arxiv.org/abs/2210.08050 · HTML: https://mlsanity.com/paper/2210.08050/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 22 references that are in the map.
- First posted on arXiv 14 Oct 2022.

## Builds on (most cited first, 9 of 9)

- [FLTrust: Byzantine-robust Federated Learning via Trust Bootstrapping](https://mlsanity.com/paper/2012.13995/) (2020)
- [Interactive Learning from Policy-Dependent Human Feedback](https://mlsanity.com/paper/1701.06049/) (2017)
- [The EMPATHIC Framework for Task Learning from Implicit Human Feedback](https://mlsanity.com/paper/2009.13649/) (2020)
- [AC-Teach: A Bayesian Actor-Critic Method for Policy Learning with an Ensemble of Suboptimal Teachers](https://mlsanity.com/paper/1909.04121/) (2019)
- [An Optimal Online Method of Selecting Source Policies for Reinforcement Learning](https://mlsanity.com/paper/1709.08201/) (2017)
- [Federated Learning with Unreliable Clients: Performance Analysis and Mechanism Design](https://mlsanity.com/paper/2105.06256/) (2021)
- [Theoretically-Grounded Policy Advice from Multiple Teachers in Reinforcement Learning Settings with Applications to Negative Transfer](https://mlsanity.com/paper/1604.03986/) (2016)
- [Persistent Rule-based Interactive Reinforcement Learning](https://mlsanity.com/paper/2102.02441/) (2021)
- [Stochastic Collection and Replenishment (SCAR) Optimisation for Persistent Autonomy](https://mlsanity.com/paper/1603.01932/) (2016)

## Built on by (most cited first, 2 of 2)

- [MADDM: Multi-Advisor Dynamic Binary Decision-Making by Maximizing the Utility](https://mlsanity.com/paper/2305.08664/) (2023)
- [Learning Safe Agent Behaviour from Human Preferences and Justifications via World Models](https://mlsanity.com/paper/2607.13172/) (2026)

## Abstract (arXiv)

Interactive reinforcement learning can effectively facilitate the agent training via human feedback. However, such methods often require the human teacher to know what is the correct action that the agent should take. In other words, if the human teacher is not always reliable, then it will not be consistently able to guide the agent through its training. In this paper, we propose a more effective interactive reinforcement learning system by introducing multiple trainers, namely Multi-Trainer Interactive Reinforcement Learning (MTIRL), which could aggregate the binary feedback from multiple non-perfect trainers into a more reliable reward for an agent training in a reward-sparse environment. In particular, our trainer feedback aggregation experiments show that our aggregation method has the best accuracy when compared with the majority voting, the weighted voting, and the Bayesian method. Finally, we conduct a grid-world experiment to show that the policy trained by the MTIRL with the review model is closer to the optimal policy than that without a review model.