# Multi-Advisor Reinforcement Learning

arXiv: https://arxiv.org/abs/1704.00756 · HTML: https://mlsanity.com/paper/1704.00756/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Apr 2017.

## Built on by (most cited first, 7 of 7)

- [Consistent Aggregation of Objectives with Diverse Time Preferences Requires Non-Markovian Rewards](https://mlsanity.com/paper/2310.00435/) (2023)
- [Value Function Decomposition for Iterative Design of Reinforcement Learning Agents](https://mlsanity.com/paper/2206.13901/) (2022)
- [Prioritized Soft Q-Decomposition for Lexicographic Reinforcement Learning](https://mlsanity.com/paper/2310.02360/) (2023)
- [Orchestrated Value Mapping for Reinforcement Learning](https://mlsanity.com/paper/2203.07171/) (2022)
- [Successor Features for Efficient Multisubject Controlled Text Generation](https://mlsanity.com/paper/2311.04921/) (2023)
- [On the Value of Myopic Behavior in Policy Reuse](https://mlsanity.com/paper/2305.17623/) (2023)
- [Progress Constraints for Reinforcement Learning in Behavior Trees](https://mlsanity.com/paper/2602.06525/) (2026)

## Abstract (arXiv)

We consider tackling a single-agent RL problem by distributing it to n learners. These learners, called advisors, endeavour to solve the problem from a different focus. Their advice, taking the form of action values, is then communicated to an aggregator, which is in control of the system. We show that the local planning method for the advisors is critical and that none of the ones found in the literature is flawless: the egocentric planning overestimates values of states where the other advisors disagree, and the agnostic planning is inefficient around danger zones. We introduce a novel approach called empathic and discuss its theoretical aspects. We empirically examine and validate our theoretical findings on a fruit collection task.