mlsanityData to 14 Oct 2026

Agnostic Interactive Imitation Learning: New Theory and Practical Algorithms

arXiv 2312.16860 ↗First posted 28 Dec 2023cs.LGcited by 1

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

We study interactive imitation learning, where a learner interactively queries a demonstrating expert for action annotations, aiming to learn a policy that has performance competitive with the expert, using as few annotations as possible. We focus on the general agnostic setting where the expert demonstration policy may not be contained in the policy class used by the learner. We propose a new oracle-efficient algorithm MFTPL-P (abbreviation for Mixed Follow the Perturbed Leader with Poisson perturbations) with provable finite-sample guarantees, under the assumption that the learner is given access to samples from some ``explorative'' distribution over states. Our guarantees hold for any policy class, which is considerably broader than prior state of the art. We further propose Bootstrap-Dagger, a more practical variant that does not require additional sample access. Empirically, MFTPL-P and Bootstrap-Dagger notably surpass online and offline imitation learning baselines in continuous control tasks.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 39 of its 68 references that are in the map. First posted on arXiv 28 Dec 2023.

Builds on

39 of 68 references.

Closest ancestors · 16

Same direction, newest first

  1. 2023Selective Sampling and Imitation Learning via Online Regression11 Jul
  2. 2022Interactive Imitation Learning in Robotics: A Survey31 Oct
  3. 2022On Efficient Online Imitation Learning via Classification26 Sep
  4. 2022Minimax Optimal Online Imitation Learning via Replay Estimation30 May
  5. 2021ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learning17 Sep
  6. 2021Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gap4 Mar
  7. 2021Feedback in Imitation Learning: The Three Regimes of Covariate Shift4 Feb
  8. 2019Imitation Learning as f-Divergence Minimization30 May
  9. 2019Provably Efficient Imitation Learning from Observation Alone27 May
  10. 2019Uncertainty-Aware Data Aggregation for Deep Imitation Learning7 May
  11. 2018An Algorithmic Perspective on Imitation Learning16 Nov
  12. 2018Predictor-Corrector Policy Optimization15 Oct
4 more
  1. 2018HG-DAgger: Interactive Imitation Learning with Human Experts5 Oct
  2. 2018EnsembleDAgger: A Bayesian Approach to Safe Imitation Learning22 Jul
  3. 2018Accelerating Imitation Learning with Predictive Models12 Jun
  4. 2018Convergence of Value Aggregation for Imitation Learning22 Jan

Other references · 23

  1. 2015Trust Region Policy Optimizationcited by 2,580
  2. 2016Generative Adversarial Imitation Learningcited by 1,597
  3. 2016End to End Learning for Self-Driving Carscited by 1,009
  4. 2019Information-Theoretic Considerations in Batch Reinforcement Learningcited by 287
  5. 2017DART: Noise Injection for Robust Imitation Learningcited by 191
  6. 2014Reinforcement and Imitation Learning via Interactive No-Regret Learningcited by 118
  7. 2020Bypassing the Monster: A Faster and Simpler Optimal Algorithm for Contextual Bandits under Realizabilitycited by 115
  8. 2020Batch Value-function Approximation with Only Realizabilitycited by 104
  9. 2017Deeply AggreVaTeD: Differentiable Imitation Learning for Sequential Predictioncited by 93
  10. 2016Query-Efficient Imitation Learning for End-to-End Autonomous Drivingcited by 84
  11. 2022The Role of Coverage in Online Reinforcement Learningcited by 83
  12. 2018On Oracle-Efficient PAC RL with Rich Observationscited by 71
11 more
  1. 2021Smoothed Analysis with Adaptive Adversariescited by 57
  2. 2022Smoothed Online Learning is as Easy as Statistical Learningcited by 52
  3. 2015The Computational Power of Optimization in Online Learningcited by 39
  4. 2016Efficient Algorithms for Adversarial Contextual Learningcited by 33
  5. 2016Oracle-Efficient Online Learning and Auction Designcited by 33
  6. 2014Online Linear Optimization via Smoothingcited by 24
  7. 2022Oracle-Efficient Online Learning for Beyond Worst-Case Adversariescited by 21
  8. 2016BISTRO: An Efficient Relaxation-Based Method for Contextual Banditscited by 20
  9. 2017DropoutDAgger: A Bayesian Approach to Safe Imitation Learningcited by 14
  10. 2020Concentration Inequalities for Multinoulli Random Variablescited by 12
  11. 2023When is Agnostic Reinforcement Learning Statistically Tractable?cited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Behavior Cloning is Not All You Need: The Optimality of On-Policy Distillation for Noisy Expert Feedbackcited by 4

Similar, not cited

Close in wording, no citation either way

  1. 2024Interactive and Hybrid Imitation Learning: Provably Beating Behavior Cloning
  2. 2025The Pitfalls of Imitation Learning when Actions are Continuous
  3. 2022Imitation Learning by Estimating Expertise of Demonstrators
  4. 2025Sample-Efficient Expert Query Control in Active Imitation Learning via Conformal Prediction
  5. 2023RLIF: Interactive Imitation Learning as Reinforcement Learning
  6. 2023MEGA-DAgger: Imitation Learning with Multiple Imperfect Experts
  7. 2023A Strong Baseline for Batch Imitation Learning
  8. 2026Difference-Aware Retrieval Policies for Imitation Learning