# Agent-Agnostic Human-in-the-Loop Reinforcement Learning

arXiv: https://arxiv.org/abs/1701.04079 · HTML: https://mlsanity.com/paper/1701.04079/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jan 2017.

## Built on by (most cited first, 23 of 23)

- [Scalable agent alignment via reward modeling: a research direction](https://mlsanity.com/paper/1811.07871/) (2018)
- [LLM-based Multi-Agent Reinforcement Learning: Current and Future Directions](https://mlsanity.com/paper/2405.11106/) (2024)
- [Efficient Learning of Safe Driving Policy via Human-AI Copilot Optimization](https://mlsanity.com/paper/2202.10341/) (2022)
- [Safe Driving via Expert Guided Policy Optimization](https://mlsanity.com/paper/2110.06831/) (2021)
- [Learning from Active Human Involvement through Proxy Value Propagation](https://mlsanity.com/paper/2502.03369/) (2025)
- [Diagnosis, Feedback, Adaptation: A Human-in-the-Loop Framework for Test-Time Policy Adaptation](https://mlsanity.com/paper/2307.06333/) (2023)
- [Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Tasks](https://mlsanity.com/paper/2107.05825/) (2021)
- [Preference-Conditioned Language-Guided Abstraction](https://mlsanity.com/paper/2402.03081/) (2024)
- [How RL Agents Behave When Their Actions Are Modified](https://mlsanity.com/paper/2102.07716/) (2021)
- [Guarded Policy Optimization with Imperfect Online Demonstrations](https://mlsanity.com/paper/2303.01728/) (2023)
- [Shattering the Agent-Environment Interface for Fine-Tuning Inclusive Language Models](https://mlsanity.com/paper/2305.11455/) (2023)
- [Human-In-The-Loop Machine Learning for Safe and Ethical Autonomous Vehicles: Principles, Challenges, and Opportunities](https://mlsanity.com/paper/2408.12548/) (2024)
- [Think Twice, Act Once: A Co-Evolution Framework of LLM and RL for Large-Scale Decision Making](https://mlsanity.com/paper/2506.02522/) (2025)
- [DIAMBRA Arena: a New Reinforcement Learning Platform for Research and Experimentation](https://mlsanity.com/paper/2210.10595/) (2022)
- [Data-Efficient Learning from Human Interventions for Mobile Robots](https://mlsanity.com/paper/2503.04969/) (2025)
- [Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Review](https://mlsanity.com/paper/2411.10268/) (2024)
- [Teachable Reinforcement Learning via Advice Distillation](https://mlsanity.com/paper/2203.11197/) (2022)
- [Human Decision Makings on Curriculum Reinforcement Learning with Difficulty Adjustment](https://mlsanity.com/paper/2208.02932/) (2022)
- [Bounded Autonomy for Enterprise AI: Typed Action Contracts and Consumer-Side Execution](https://mlsanity.com/paper/2604.14723/) (2026)
- [A Reinforcement Learning-based Offensive semantics Censorship System for Chatbots](https://mlsanity.com/paper/2207.10569/) (2022)
- [Computational Charisma -- A Brick by Brick Blueprint for Building Charismatic Artificial Intelligence](https://mlsanity.com/paper/2301.00142/) (2022)
- [Constrained Exploration in Reinforcement Learning with Optimality Preservation](https://mlsanity.com/paper/2304.03104/) (2023)
- [Can you see how I learn? Human observers' inferences about Reinforcement Learning agents' learning processes](https://mlsanity.com/paper/2506.13583/) (2025)

## Abstract (arXiv)

Providing Reinforcement Learning agents with expert advice can dramatically improve various aspects of learning. Prior work has developed teaching protocols that enable agents to learn efficiently in complex environments; many of these methods tailor the teacher's guidance to agents with a particular representation or underlying learning scheme, offering effective but specialized teaching procedures. In this work, we explore protocol programs, an agent-agnostic schema for Human-in-the-Loop Reinforcement Learning. Our goal is to incorporate the beneficial properties of a human teacher into Reinforcement Learning without making strong assumptions about the inner workings of the agent. We show how to represent existing approaches such as action pruning, reward shaping, and training in simulation as special cases of our schema and conduct preliminary experiments on simple domains.