mlsanityData to 7 Oct 2026

When to Ask for Help: Proactive Interventions in Autonomous Reinforcement Learning

arXiv 2210.10765 ↗First posted 19 Oct 2022cs.LGcited by 20

Not on the map. Why?

Abstract (arXiv)

A long-term goal of reinforcement learning is to design agents that can autonomously interact and learn in the world. A critical challenge to such autonomy is the presence of irreversible states which require external assistance to recover from, such as when a robot arm has pushed an object off of a table. While standard agents require constant monitoring to decide when to intervene, we aim to design proactive agents that can request human intervention only when needed. To this end, we propose an algorithm that efficiently learns to detect and avoid states that are irreversible, and proactively asks for help in case the agent does enter them. On a suite of continuous control environments with unknown irreversible states, we find that our algorithm exhibits better sample- and intervention-efficiency compared to existing methods. Our code is publicly available at https://sites.google.com/view/proactive-interventions

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 39 of its 56 references that are in the map. First posted on arXiv 19 Oct 2022.

Builds on

39 of 56 references.

References · 39

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2017Constrained Policy Optimizationcited by 928
  3. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  4. 2017Safe Reinforcement Learning via Shieldingcited by 427
  5. 2018Reward Constrained Policy Optimizationcited by 397
  6. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  7. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 342
  8. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 320
  9. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 298
  10. 2020Never Give Up: Learning Directed Exploration Strategiescited by 210
  11. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 156
  12. 2017Interactive Learning from Policy-Dependent Human Feedbackcited by 155
27 more
  1. 2015Deep Spatial Autoencoders for Visuomotor Learningcited by 149
  2. 2020The Ingredients of Real-World Robotic Reinforcement Learningcited by 111
  3. 2018Episodic Curiosity through Reachabilitycited by 109
  4. 2020Learning to be Safe: Deep RL with a Safety Criticcited by 109
  5. 2020Conservative Safety Critics for Explorationcited by 107
  6. 2021ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learningcited by 98
  7. 2018EnsembleDAgger: A Bayesian Approach to Safe Imitation Learningcited by 96
  8. 2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 90
  9. 2018Batch Active Preference-Based Learning of Reward Functionscited by 87
  10. 2017Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement Learningcited by 86
  11. 2016Query-Efficient Imitation Learning for End-to-End Autonomous Drivingcited by 83
  12. 2018Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Costcited by 79
  13. 2020Safe Reinforcement Learning via Curriculum Inductioncited by 73
  14. 2022Safe Reinforcement Learning by Imagining the Near Futurecited by 67
  15. 2021LazyDAgger: Reducing Context Switching in Interactive Imitation Learningcited by 50
  16. 2021Safe Reinforcement Learning Using Advantage-Based Interventioncited by 45
  17. 2021A Survey on Interactive Reinforcement Learning: Design Principles and Open Challengescited by 41
  18. 2018Penalizing side effects using stepwise relative reachabilitycited by 38
  19. 2021Skill Preferences: Learning to Extract and Execute Robotic Skills from Human Feedbackcited by 34
  20. 2021Autonomous Reinforcement Learning: Formalism and Benchmarkingcited by 33
  21. 2021Autonomous Reinforcement Learning via Subgoal Curriculacited by 32
  22. 2017Deep Predictive Policy Training using Reinforcement Learningcited by 30
  23. 2020Continual Learning of Control Primitives: Skill Discovery via Reset-Gamescited by 29
  24. 2022A State-Distribution Matching Approach to Non-Episodic Reinforcement Learningcited by 21
  25. 2021Safe Continuous Control with Constrained Model-Based Policy Optimizationcited by 17
  26. 2021There Is No Turning Back: A Self-Supervised Approach for Reversibility-Aware Reinforcement Learningcited by 17
  27. 2018Interactive Reinforcement Learning with Dynamic Reuse of Prior Knowledge from Human/Agent's Demonstrationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 5; 2025: 6; 2026: 4.

20 papers.

Sort:

By area:

  1. 2024Preference Fine-Tuning of LLMs Should Leverage Suboptimal, On-Policy Datacited by 184
  2. 2024SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learningcited by 113
  3. 2025SAFE: Multitask Failure Detection for Vision-Language-Action Modelscited by 74
  4. 2023Model-Based Runtime Monitoring with Interactive Imitation Learningcited by 37
  5. 2023Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learningcited by 29
  6. 2024Towards Explainable, Safe Autonomous Driving with Language Embeddings for Novelty Identification and Active Learning: Framework and Experimental Analysis with Real-World Data Setscited by 25
  7. 2024Tools Fail: Detecting Silent Errors in Faulty Toolscited by 23
  8. 2026CycleVLA: Proactive Self-Correcting Vision-Language-Action Models via Subtask Backtracking and Minimum Bayes Risk Decodingcited by 13
  9. 2023Decision Making for Human-in-the-loop Robotic Agents via Uncertainty-Aware Reinforcement Learningcited by 9
  10. 2025Safety Representations for Safer Policy Learningcited by 4
  11. 2026UniIntervene: Agentic Intervention for Efficient Real-World Reinforcement Learningcited by 4
  12. 2023When Learning Is Out of Reach, Reset: Generalization in Autonomous Visuomotor Reinforcement Learningcited by 3
8 more
  1. 2023Demonstration-free Autonomous Reinforcement Learning via Implicit and Bidirectional Curriculumcited by 2
  2. 2026SAFECAST: Robust Failure Detection for VLA Policies with Contrast-Set Training and Calibrationcited by 2
  3. 2026Temporal Difference Calibration in Sequential Tasks: Application to Vision-Language-Action Modelscited by 1
  4. 2024Language-guided Robust Navigation for Mobile Robots in Dynamically-changing Environmentscited by 0
  5. 2025YRC-Bench: A Benchmark for Learning to Coordinate with Expertscited by 0
  6. 2025EAPO: Enhancing Policy Optimization with On-Demand Expert Assistancecited by 0
  7. 2025Approximating Safety Feedback Without a Safety Oracle via Model Predictive Controlcited by 0
  8. 2025Learning Safe Autonomous Driving Policies Using Predictive Safety Representationscited by 0