mlsanityData to 7 Oct 2026

ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learning

arXiv 2109.08273 ↗First posted 17 Sep 2021cs.RO cs.AIcited by 98

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Effective robot learning often requires online human feedback and interventions that can cost significant human time, giving rise to the central challenge in interactive imitation learning: is it possible to control the timing and length of interventions to both facilitate learning and limit burden on the human supervisor? This paper presents ThriftyDAgger, an algorithm for actively querying a human supervisor given a desired budget of human interventions. ThriftyDAgger uses a learned switching policy to solicit interventions only at states that are sufficiently (1) novel, where the robot policy has no reference behavior to imitate, or (2) risky, where the robot has low confidence in task completion. To detect the latter, we introduce a novel metric for estimating risk under the current robot policy. Experiments in simulation and on a physical cable routing experiment suggest that ThriftyDAgger's intervention criteria balances task performance and supervisor burden more effectively than prior algorithms. ThriftyDAgger can also be applied at execution time, where it achieves a 100% success rate on both the simulation and physical tasks. A user study (N=10) in which users control a three-robot fleet while also performing a concentration task suggests that ThriftyDAgger increases human and robot performance by 58% and 80% respectively compared to the next best algorithm while reducing supervisor burden.

Key facts. Cited by 98 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 27 of its 52 references that are in the map. First posted on arXiv 17 Sep 2021.

Builds on

27 of 52 references.

Closest ancestors · 7

Same direction, newest first

  1. 2020Interactive Imitation Learning in State-Space2 Aug
  2. 2020Safe Imitation Learning via Fast Bayesian Reward Inference from Preferences21 Feb
  3. 2019On-Policy Robot Imitation Learning from a Converging Supervisor8 Jul
  4. 2018An Algorithmic Perspective on Imitation Learning16 Nov
  5. 2018HG-DAgger: Interactive Imitation Learning with Human Experts5 Oct
  6. 2018EnsembleDAgger: A Bayesian Approach to Safe Imitation Learning22 Jul
  7. 2018Behavioral Cloning from Observation4 May

Other references · 20

  1. 2017Deep reinforcement learning from human preferencescited by 3,846
  2. 2016Generative Adversarial Imitation Learningcited by 1,576
  3. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  4. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  5. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  6. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  7. 2018Reward learning from human preferences and demonstrations in Ataricited by 314
  8. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progresscited by 294
  9. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  10. 2017DART: Noise Injection for Robust Imitation Learningcited by 187
  11. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 156
  12. 2019A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithmscited by 136
8 more
  1. 2014Reinforcement and Imitation Learning via Interactive No-Regret Learningcited by 116
  2. 2020Human-in-the-Loop Imitation Learning using Remote Teleoperationcited by 103
  3. 2016Query-Efficient Imitation Learning for End-to-End Autonomous Drivingcited by 83
  4. 2019Asking Easy Questions: A User-Friendly Approach to Active Reward Learningcited by 67
  5. 2018Shared Autonomy via Deep Reinforcement Learningcited by 65
  6. 2019Learning Reward Functions by Integrating Human Demonstrations and Preferencescited by 64
  7. 2021LazyDAgger: Reducing Context Switching in Interactive Imitation Learningcited by 50
  8. 2019Scaled Autonomy: Enabling Human Operators to Control Robot Fleetscited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 10; 2023: 22; 2024: 15; 2025: 19; 2026: 31.

98 papers.

Sort:

By area:

  1. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  2. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 308
  3. 2024RT-H: Action Hierarchies Using Languagecited by 204
  4. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 145
  5. 2024Yell At Your Robot: Improving On-the-Fly from Language Correctionscited by 123
  6. 2023Motion Planning for Autonomous Driving: The State of the Art and Future Perspectivescited by 112
  7. 2022Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deploymentcited by 101
  8. 2023Data Quality in Imitation Learningcited by 72
  9. 2023Imitating Task and Motion Planning with Visuomotor Transformerscited by 63
  10. 2024Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progresscited by 63
  11. 2023"No, to the Right" -- Online Language Corrections for Robotic Manipulation via Shared Autonomycited by 60
  12. 2023NeRF in the Palm of Your Hand: Corrective Augmentation for Robotics via Novel-View Synthesiscited by 56
38 more
  1. 2023Benchmarks and Algorithms for Offline Preference-Based Reward Learningcited by 55
  2. 2023HYDRA: Hybrid Robot Actions for Imitation Learningcited by 54
  3. 2024TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correctioncited by 50
  4. 2024Bidirectional Decoding: Improving Action Chunking via Guided Test-Time Samplingcited by 46
  5. 2022Efficient Learning of Safe Driving Policy via Human-AI Copilot Optimizationcited by 44
  6. 2023Model-Based Runtime Monitoring with Interactive Imitation Learningcited by 37
  7. 2024Multi-Task Interactive Robot Fleet Learning with Visual World Modelscited by 33
  8. 2022Fleet-DAgger: Interactive Robot Fleet Learning with Scalable Human Supervisioncited by 29
  9. 2024Diff-DAgger: Uncertainty Estimation with Diffusion Policy for Robotic Manipulationcited by 29
  10. 2023Human-in-the-Loop Task and Motion Planning for Imitation Learningcited by 28
  11. 2023Interactive Robot Learning from Verbal Correctioncited by 28
  12. 2025Compliant Residual DAgger: Improving Real-World Contact-Rich Manipulation with Human Correctionscited by 25
  13. 2023RLIF: Interactive Imitation Learning as Reinforcement Learningcited by 24
  14. 2024IntervenGen: Interventional Data Generation for Robust and Data-Efficient Robot Imitation Learningcited by 24
  15. 2022Play it by Ear: Learning Skills amidst Occlusion through Audio-Visual Imitation Learningcited by 22
  16. 2022Eliciting Compatible Demonstrations for Multi-Human Imitation Learningcited by 22
  17. 2025RoboCopilot: Human-in-the-loop Interactive Imitation Learning for Robot Manipulationcited by 22
  18. 2022When to Ask for Help: Proactive Interventions in Autonomous Reinforcement Learningcited by 20
  19. 2022PATO: Policy Assisted TeleOperation for Scalable Robot Data Collectioncited by 19
  20. 2022Unified Learning from Demonstrations, Corrections, and Preferences during Physical Human-Robot Interactioncited by 18
  21. 2026When to Trust Imagination: Adaptive Action Execution for World Action Modelscited by 18
  22. 2023MEGA-DAgger: Imitation Learning with Multiple Imperfect Expertscited by 17
  23. 2024So You Think You Can Scale Up Autonomous Robot Data Collection?cited by 17
  24. 2024SAFE-GIL: SAFEty Guided Imitation Learning for Robotic Systemscited by 13
  25. 2023Active Reward Learning from Online Preferencescited by 12
  26. 2025A Survey on Imitation Learning for Contact-Rich Tasks in Roboticscited by 12
  27. 2022A Benchmark Comparison of Imitation Learning-based Control Policies for Autonomous Racingcited by 11
  28. 2024Conformalized Interactive Imitation Learning: Handling Expert Shift and Intermittent Feedbackcited by 11
  29. 2023How To Guide Your Learner: Imitation Learning with Active Adaptive Expert Involvementcited by 8
  30. 2023BEDD: The MineRL BASALT Evaluation and Demonstrations Dataset for Training and Benchmarking Agents that Solve Fuzzy Taskscited by 8
  31. 2026DeMaVLA: A Vision-Language-Action Foundation Model for Generalizable Deformable Manipulationcited by 8
  32. 2025MILE: Model-based Intervention Learningcited by 7
  33. 2025Genie Centurion: Accelerating Scalable Real-World Robot Training with Human Rewind-and-Refine Guidancecited by 7
  34. 2025Dynamic Rank Adjustment in Diffusion Policies for Efficient and Flexible Trainingcited by 6
  35. 2025Uncertainty Comes for Free: Human-in-the-Loop Policies with Diffusion Modelscited by 6
  36. 2025Robot-Gated Interactive Imitation Learning with Adaptive Intervention Mechanismcited by 6
  37. 2024Leveraging Demonstrator-perceived Precision for Safe Interactive Imitation Learning of Clearance-limited Taskscited by 5
  38. 2026DexHiL: A Human-in-the-Loop Framework for Vision-Language-Action Model Post-Training in Dexterous Manipulationcited by 5

50 most cited of 98 All 98 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025SafeMIL: Learning Offline Safe Imitation Policy from Non-Preferred Trajectories
  2. 2026Reward-DAgger: Robot-Gated Interactive Imitation Learning with General-Purpose Progress-Based Reward Models
  3. 2025Using Non-Expert Data to Robustify Imitation Learning via Offline Reinforcement Learning
  4. 2025Robust Instant Policy: Leveraging Student's t-Regression Model for Robust In-context Imitation Learning of Robot Manipulation
  5. 2023A Strong Baseline for Batch Imitation Learning
  6. 2025Steering Robots with Inference-Time Interactions
  7. 2024Policy Decorator: Model-Agnostic Online Refinement for Large Policy Model
  8. 2022Imitation Learning by Estimating Expertise of Demonstrators