mlsanityData to 7 Oct 2026

MILE: Model-based Intervention Learning

arXiv 2502.13519 ↗First posted 19 Feb 2025cs.RO cs.AI cs.LGcited by 7

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning techniques have been shown to be highly effective in real-world control scenarios, such as robotics. However, these approaches not only suffer from compounding error issues but also require human experts to provide complete trajectories. Although there exist interactive methods where an expert oversees the robot and intervenes if needed, these extensions usually only utilize the data collected during intervention periods and ignore the feedback signal hidden in non-intervention timesteps. In this work, we create a model to formulate how the interventions occur in such cases, and show that it is possible to learn a policy with just a handful of expert interventions. Our key insight is that it is possible to get crucial information about the quality of the current state and the optimality of the chosen action from expert feedback, regardless of the presence or the absence of intervention. We evaluate our method on various discrete and continuous simulation environments, a real-world robotic manipulation task, as well as a human subject study. Videos and the code can be found at https://liralab.usc.edu/mile .

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 31 of its 48 references that are in the map. First posted on arXiv 19 Feb 2025.

Builds on

31 of 48 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023RLIF: Interactive Imitation Learning as Reinforcement Learning21 Nov
  2. 2023RoboCLIP: One Demonstration is Enough to Learn Robot Policies11 Oct
  3. 2022Interactive Imitation Learning in Robotics: A Survey31 Oct
  4. 2021ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learning17 Sep
  5. 2018HG-DAgger: Interactive Imitation Learning with Human Experts5 Oct
  6. 2018EnsembleDAgger: A Bayesian Approach to Safe Imitation Learning22 Jul

Other references · 25

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2016Generative Adversarial Imitation Learningcited by 1,576
  3. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  4. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  5. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  6. 2023Efficient Online Reinforcement Learning with Offline Datacited by 332
  7. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  8. 2017DART: Noise Injection for Robust Imitation Learningcited by 187
  9. 2021Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learningcited by 157
  10. 2024Yell At Your Robot: Improving On-the-Fly from Language Correctionscited by 123
  11. 2020Human-in-the-Loop Imitation Learning using Remote Teleoperationcited by 103
  12. 2017Trial without Error: Towards Safe Reinforcement Learning via Human Interventioncited by 102
13 more
  1. 2022Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deploymentcited by 101
  2. 2024TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correctioncited by 50
  3. 2015Shared Autonomy via Hindsight Optimizationcited by 46
  4. 2022Efficient Learning of Safe Driving Policy via Human-AI Copilot Optimizationcited by 44
  5. 2020When Humans Aren't Optimal: Robots that Collaborate with Risk-Aware Humanscited by 34
  6. 2022The Boltzmann Policy Distribution: Accounting for Systematic Suboptimality in Human Modelscited by 31
  7. 2022Fleet-DAgger: Interactive Robot Fleet Learning with Scalable Human Supervisioncited by 29
  8. 2021Correct Me if I am Wrong: Interactive Learning for Robotic Manipulationcited by 24
  9. 2024IntervenGen: Interventional Data Generation for Robust and Data-Efficient Robot Imitation Learningcited by 24
  10. 2024Trajectory Improvement and Reward Learning from Comparative Language Feedbackcited by 17
  11. 2020Modeling and Predicting Trust Dynamics in Human-Robot Teaming: A Bayesian Inference Approachcited by 14
  12. 2023Active Reward Learning from Online Preferencescited by 12
  13. 2025Effects of Robot Competency and Motion Legibility on Human Correction Feedbackcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.

  1. 2026Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisonscited by 51
  2. 2025ADAPT: Actively Discovering and Adapting to Preferences for any Taskcited by 5
  3. 2026Causally Robust Reward Learning from Reason-Augmented Preference Feedbackcited by 2
  4. 2025Residual Reward Models: Leveraging Prior Knowledge for Efficient Preference-based Reinforcement Learning in Roboticscited by 1
  5. 2025Data-Efficient Multitask DAggercited by 0
  6. 2026The Geometry of Learning to Avoid Interventionscited by 0
  7. 2026Easy-IIL: Reducing Human Operational Burden in Interactive Imitation Learning via Assistant Expertscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024MILES: Making Imitation Learning Easy with Self-Supervision
  2. 2023Model-Based Runtime Monitoring with Interactive Imitation Learning
  3. 2022Imitation Learning by Estimating Expertise of Demonstrators
  4. 2025Robot-Gated Interactive Imitation Learning with Adaptive Intervention Mechanism
  5. 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations
  6. 2022Robust Learning from Observation with Model Misspecification
  7. 2025Self-Augmented Robot Trajectory: Efficient Imitation Learning via Safe Self-augmentation with Demonstrator-annotated Precision
  8. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges