# Robot-Gated Interactive Imitation Learning with Adaptive Intervention Mechanism

arXiv: https://arxiv.org/abs/2506.09176 · HTML: https://mlsanity.com/paper/2506.09176/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 20 of its 30 references that are in the map.
- First posted on arXiv 10 Jun 2025.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 20 of 20)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [An Algorithmic Perspective on Imitation Learning](https://mlsanity.com/paper/1811.06711/) (2018)
- [MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning](https://mlsanity.com/paper/2109.12674/) (2021)
- [HG-DAgger: Interactive Imitation Learning with Human Experts](https://mlsanity.com/paper/1810.02890/) (2018)
- [One-Shot Visual Imitation Learning via Meta-Learning](https://mlsanity.com/paper/1709.04905/) (2017)
- [A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges](https://mlsanity.com/paper/2309.02473/) (2023)
- [Human-in-the-Loop Imitation Learning using Remote Teleoperation](https://mlsanity.com/paper/2012.06733/) (2020)
- [ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learning](https://mlsanity.com/paper/2109.08273/) (2021)
- [EnsembleDAgger: A Bayesian Approach to Safe Imitation Learning](https://mlsanity.com/paper/1807.08364/) (2018)
- [Query-Efficient Imitation Learning for End-to-End Autonomous Driving](https://mlsanity.com/paper/1605.06450/) (2016)
- [Contrastive Preference Learning: Learning from Human Feedback without RL](https://mlsanity.com/paper/2310.13639/) (2023)
- [Shared Autonomy via Deep Reinforcement Learning](https://mlsanity.com/paper/1802.01744/) (2018)
- [LazyDAgger: Reducing Context Switching in Interactive Imitation Learning](https://mlsanity.com/paper/2104.00053/) (2021)
- [Efficient Learning of Safe Driving Policy via Human-AI Copilot Optimization](https://mlsanity.com/paper/2202.10341/) (2022)
- [Safe Driving via Expert Guided Policy Optimization](https://mlsanity.com/paper/2110.06831/) (2021)
- [RLIF: Interactive Imitation Learning as Reinforcement Learning](https://mlsanity.com/paper/2311.12996/) (2023)
- [Interactive Policy Learning through Confidence-Based Autonomy](https://mlsanity.com/paper/1401.3439/) (2014)
- [Learning from Active Human Involvement through Proxy Value Propagation](https://mlsanity.com/paper/2502.03369/) (2025)
- [Efficient Active Imitation Learning with Random Network Distillation](https://mlsanity.com/paper/2411.01894/) (2024)
- [Data-Efficient Learning from Human Interventions for Mobile Robots](https://mlsanity.com/paper/2503.04969/) (2025)

## Built on by (most cited first, 9 of 9)

- [Predictive Preference Learning from Human Interventions](https://mlsanity.com/paper/2510.01545/) (2025)
- [OHP-RL: Online Human Preference as Guidance in Reinforcement Learning for Robot Manipulation](https://mlsanity.com/paper/2605.15971/) (2026)
- [DASH-OPD: Discrepancy-Aware Switching with Hysteresis for On-Policy Distillation](https://mlsanity.com/paper/2607.29078/) (2026)
- [Hierarchical Advantage Weighting for Online RL Fine-Tuning of VLAs from Sparse Episode Outcomes](https://mlsanity.com/paper/2606.17043/) (2026)
- [HIL-UMI: Bringing Human-in-the-Loop Post-Training of Vision-Language-Action Models to Universal Manipulation Interface](https://mlsanity.com/paper/2609.20659/) (2026)
- [From Imitation to Alignment: Human-Preference Flow Policies for Long-Horizon Sidewalk Navigation](https://mlsanity.com/paper/2606.12603/) (2026)
- [No Free Checker: A Survey of Verifiers for Robot Policies](https://mlsanity.com/paper/2609.09250/) (2026)
- [Skill-Space Shooting for Autonomous Robot Policy Improvement](https://mlsanity.com/paper/2609.38178/) (2026)
- [Reward-DAgger: Robot-Gated Interactive Imitation Learning with General-Purpose Progress-Based Reward Models](https://mlsanity.com/paper/2610.04054/) (2026)

## Abstract (arXiv)

Interactive Imitation Learning (IIL) allows agents to acquire desired behaviors through human interventions, but current methods impose high cognitive demands on human supervisors. We propose the Adaptive Intervention Mechanism (AIM), a novel robot-gated IIL algorithm that learns an adaptive criterion for requesting human demonstrations. AIM utilizes a proxy Q-function to mimic the human intervention rule and adjusts intervention requests based on the alignment between agent and human actions. By assigning high Q-values when the agent deviates from the expert and decreasing these values as the agent becomes proficient, the proxy Q-function enables the agent to assess the real-time alignment with the expert and request assistance when needed. Our expert-in-the-loop experiments reveal that AIM significantly reduces expert monitoring efforts in both continuous and discrete control tasks. Compared to the uncertainty-based baseline Thrifty-DAgger, our method achieves a 40% improvement in terms of human take-over cost and learning efficiency. Furthermore, AIM effectively identifies safety-critical states for expert assistance, thereby collecting higher-quality expert demonstrations and reducing overall expert data and environment interactions needed. Code and demo video are available at https://github.com/metadriverse/AIM.