mlsanityData to 14 Oct 2026

Decision Making for Human-in-the-loop Robotic Agents via Uncertainty-Aware Reinforcement Learning

arXiv 2303.06710 ↗First posted 12 Mar 2023cs.RO cs.AI cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

In a Human-in-the-Loop paradigm, a robotic agent is able to act mostly autonomously in solving a task, but can request help from an external expert when needed. However, knowing when to request such assistance is critical: too few requests can lead to the robot making mistakes, but too many requests can overload the expert. In this paper, we present a Reinforcement Learning based approach to this problem, where a semi-autonomous agent asks for external assistance when it has low confidence in the eventual success of the task. The confidence level is computed by estimating the variance of the return from the current state. We show that this estimate can be iteratively improved during training using a Bellman-like recursion. On discrete navigation problems with both fully- and partially-observable state information, we show that our method makes effective use of a limited budget of expert calls at run-time, despite having no access to the expert at training time.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 16 of its 27 references that are in the map. First posted on arXiv 12 Mar 2023.

Builds on

16 of 27 references.

References · 16

  1. 2015Continuous control with deep reinforcement learningcited by 3,407
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  4. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  5. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
  6. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 369
  7. 2018Randomized Prior Functions for Deep Reinforcement Learningcited by 219
  8. 2015Variational Information Maximisation for Intrinsically Motivated Reinforcement Learningcited by 144
  9. 2019A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithmscited by 142
  10. 2017Uncertainty-Aware Reinforcement Learning for Collision Avoidancecited by 88
  11. 2018DQN-TAMER: Human-in-the-Loop Reinforcement Learning with Intractable Feedbackcited by 41
  12. 2020Reinforcement learning with human advice: a surveycited by 24
4 more
  1. 2022When to Ask for Help: Proactive Interventions in Autonomous Reinforcement Learningcited by 21
  2. 2021Ensemble Quantile Networks: Uncertainty-Aware Reinforcement Learning with Applications in Autonomous Drivingcited by 17
  3. 2021Human-in-the-Loop Deep Reinforcement Learning with Application to Autonomous Drivingcited by 16
  4. 2023Active Reward Learning from Multiple Teacherscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 4; 2026: 2.

9 papers.

  1. 2025Uncertainty Comes for Free: Human-in-the-Loop Policies with Diffusion Modelscited by 8
  2. 2023Skill-Critic: Refining Learned Skills for Hierarchical Reinforcement Learningcited by 6
  3. 2025Simulating the Unseen: Crash Prediction Must Learn from What Did Not Happencited by 5
  4. 2026MINT: Minimal Information Neuro-Symbolic Tree for Objective-Driven Knowledge-Gap Reasoning and Active Elicitationcited by 4
  5. 2025Enhancing Robot Navigation Policies with Task-Specific Uncertainty Managementscited by 2
  6. 2025Grower-in-the-Loop Interactive Reinforcement Learning for Greenhouse Climate Controlcited by 2
  7. 2024HARP: Human-Assisted Regrouping with Permutation Invariant Critic for Multi-Agent Reinforcement Learningcited by 1
  8. 2024Language-guided Robust Navigation for Mobile Robots in Dynamically-changing Environmentscited by 0
  9. 2026Policy-Calibrated DAgger: Offline Calibrated Noise Injection for Imitation Learningcited by 0