# UniIntervene++: An Adaptive Intervention Agent for Efficient Real-World Reinforcement Learning

arXiv: https://arxiv.org/abs/2610.03620 · HTML: https://mlsanity.com/paper/2610.03620/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 25 of its 33 references that are in the map.
- First posted on arXiv 2 Oct 2026.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 25 of 25)

- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Efficient Online Reinforcement Learning with Offline Data](https://mlsanity.com/paper/2302.02948/) (2023)
- [HG-DAgger: Interactive Imitation Learning with Human Experts](https://mlsanity.com/paper/1810.02890/) (2018)
- [Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning](https://mlsanity.com/paper/2410.21845/) (2024)
- [Recovery RL: Safe Reinforcement Learning with Learned Recovery Zones](https://mlsanity.com/paper/2010.15920/) (2020)
- [SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning](https://mlsanity.com/paper/2401.16013/) (2024)
- [ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learning](https://mlsanity.com/paper/2109.08273/) (2021)
- [Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment](https://mlsanity.com/paper/2211.08416/) (2022)
- [RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learning](https://mlsanity.com/paper/2510.14830/) (2025)
- [Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learning](https://mlsanity.com/paper/2303.01488/) (2023)
- [RLIF: Interactive Imitation Learning as Reinforcement Learning](https://mlsanity.com/paper/2311.12996/) (2023)
- [Back to the Manifold: Recovering from Out-of-Distribution States](https://mlsanity.com/paper/2207.08673/) (2022)
- [Failure-Aware RL: Reliable Offline-to-Online Reinforcement Learning with Self-Recovery for Real-World Manipulation](https://mlsanity.com/paper/2601.07821/) (2026)
- [LS3: Latent Space Safe Sets for Long-Horizon Visuomotor Control of Sparse Reward Iterative Tasks](https://mlsanity.com/paper/2107.04775/) (2021)
- [RecoveryChaining: Learning Local Recovery Policies for Robust Manipulation](https://mlsanity.com/paper/2410.13979/) (2024)
- [Efficient Recovery Learning using Model Predictive Meta-Reasoning](https://mlsanity.com/paper/2209.13605/) (2022)
- [Uncertainty Comes for Free: Human-in-the-Loop Policies with Diffusion Models](https://mlsanity.com/paper/2503.01876/) (2025)
- [Real-world Reinforcement Learning from Suboptimal Interventions](https://mlsanity.com/paper/2512.24288/) (2025)
- [UniIntervene: Agentic Intervention for Efficient Real-World Reinforcement Learning](https://mlsanity.com/paper/2606.12372/) (2026)
- [E2HiL: Entropy-Guided Sample Selection for Efficient Real-World Human-in-the-Loop Reinforcement Learning](https://mlsanity.com/paper/2601.19969/) (2026)
- [Learning Skills to Patch Plans Based on Inaccurate Models](https://mlsanity.com/paper/2009.13732/) (2020)
- [OHP-RL: Online Human Preference as Guidance in Reinforcement Learning for Robot Manipulation](https://mlsanity.com/paper/2605.15971/) (2026)
- [WorldSample: Closed-loop Real-robot RL with World Modelling](https://mlsanity.com/paper/2607.02431/) (2026)
- [Facet-0: A Robotic Foundation Model for Contact-Rich Precise Manipulation](https://mlsanity.com/paper/2609.01596/) (2026)
- [One Demonstration Is Enough for Real-World Robotic Reinforcement Learning](https://mlsanity.com/paper/2607.01651/) (2026)

## Abstract (arXiv)

Online reinforcement learning (RL) enables robot policies to improve through physical interaction, but the assistance they require changes as their competence evolves. Existing intervention strategies based on offline estimates or fixed decision rules can therefore become mismatched to the current policy. To address this, we propose UniIntervene++, an adaptive intervention agent that learns to allocate control between autonomous execution and heterogeneous assisted behaviors during online RL. Specifically, UniIntervene++ first formulates the evolving RL policy, trajectory correction, and a task-structured CodePolicy as Options in a unified semi-Markov decision process and learns their relative values online. Building on this, competence-adaptive intervention periodically probes the RL policy through unassisted execution, keeping control allocation responsive to its evolving capability. Finally, coupled experience learning allows assisted behaviors to improve the RL policy, whose evolving outcomes in turn reshape future intervention decisions. In this way, UniIntervene++ jointly determines when to intervene, how to intervene, and when to return control as the RL policy improves. Across five real-world manipulation tasks, UniIntervene++ achieves an average success rate of 89.67%, outperforming all baselines by at least 6 percentage points, while reducing human intervention to 0.77%, a relative reduction of at least 94.6% from the best baseline. Code is available in our hrefhttps://github.com/dannyyudong/An-Adaptive-Intervention-Agent-for-Efficient-Real-World-Reinforcement-LearningGitHub repository.