mlsanityData to 7 Oct 2026

Imitation Learning in Discounted Linear MDPs without exploration assumptions

arXiv 2405.02181 ↗First posted 3 May 2024cs.LGcited by 15

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

We present a new algorithm for imitation learning in infinite horizon linear MDPs dubbed ILARL which greatly improves the bound on the number of trajectories that the learner needs to sample from the environment. In particular, we remove exploration assumptions required in previous works and we improve the dependence on the desired accuracy ε from O(ε-5) to O(ε-4). Our result relies on a connection between imitation learning and online learning in MDPs with adversarial losses. For the latter setting, we present the first result for infinite horizon linear MDP which may be of independent interest. Moreover, we are able to provide a strengthen result for the finite horizon case where we achieve O(ε-2). Numerical experiments with linear function approximation shows that ILARL outperforms other commonly used algorithms.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 40 of its 57 references that are in the map. First posted on arXiv 3 May 2024.

Builds on

40 of 57 references.

Closest ancestors · 9

Same direction, newest first

  1. 2023Provably Efficient Adversarial Imitation Learning with Unknown Transitions11 Jun
  2. 2022On Efficient Online Imitation Learning via Classification26 Sep
  3. 2022Proximal Point Imitation Learning22 Sep
  4. 2022Minimax Optimal Online Imitation Learning via Replay Estimation30 May
  5. 2021IQ-Learn: Inverse soft-Q Learning for Imitation23 Jun
  6. 2020Toward the Fundamental Limits of Imitation Learning13 Sep
  7. 2020Primal Wasserstein Imitation Learning8 Jun
  8. 2019SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewards27 May
  9. 2018An Algorithmic Perspective on Imitation Learning16 Nov

Other references · 31

  1. 2016Generative Adversarial Imitation Learningcited by 1,576
  2. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 514
  3. 2016Contextual Decision Processes with Low Bellman Rank are PAC-Learnablecited by 290
  4. 2020FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPscited by 232
  5. 2019Provably Efficient Exploration in Policy Optimizationcited by 203
  6. 2020Nearly Minimax Optimal Reinforcement Learning for Linear Mixture Markov Decision Processescited by 199
  7. 2021Bilinear Classes: A Structural Framework for Provable Generalization in RLcited by 176
  8. 2020Minimax-Optimal Off-Policy Evaluation with Linear Function Approximationcited by 120
  9. 2021Reward (Mis)design for Autonomous Drivingcited by 102
  10. 2020Optimistic Policy Optimization with Bandit Feedbackcited by 79
  11. 2020f-IRL: Inverse Reinforcement Learning via State Marginal Matchingcited by 67
  12. 2020Reinforcement Learning in Economics and Financecited by 57
19 more
  1. 2022Nearly Minimax Optimal Reinforcement Learning for Linear Markov Decision Processescited by 57
  2. 2016Model-Free Imitation Learning with Policy Optimizationcited by 52
  3. 2021Policy Optimization in Adversarial MDPs: Improved Exploration via Dilated Bonusescited by 51
  4. 2020Near-optimal Regret Bounds for Stochastic Shortest Pathcited by 50
  5. 2022Maximum-Likelihood Inverse Reinforcement Learning with Finite-Time Guaranteescited by 42
  6. 2023Inverse Reinforcement Learning without Reinforcement Learningcited by 35
  7. 2020Robust Inverse Reinforcement Learning under Transition Dynamics Mismatchcited by 33
  8. 2021Online Apprenticeship Learningcited by 28
  9. 2021Near-optimal Policy Optimization Algorithms for Learning Adversarial Linear Mixture MDPscited by 28
  10. 2023A Theoretical Analysis of Optimistic Proximal Policy Optimization in Linear Markov Decision Processescited by 28
  11. 2020Online Sparse Reinforcement Learningcited by 25
  12. 2013Online Learning in Markov Decision Processes with Adversarially Chosen Transition Probability Distributionscited by 18
  13. 2023Refined Regret for Adversarial MDPs with Linear Function Approximationcited by 18
  14. 2023Improved Regret for Efficient Online Reinforcement Learning with Linear Function Approximationcited by 16
  15. 2023Rate-Optimal Policy Optimization for Linear Markov Decision Processescited by 14
  16. 2022Structural Estimation of Markov Decision Processes in High-Dimensional State Space with Finite-Time Guaranteescited by 12
  17. 2020A maximum-entropy approach to off-policy evaluation in average-reward MDPscited by 9
  18. 2023Optimistic Planning by Regularized Dynamic Programmingcited by 8
  19. 2023Inverse Reinforcement Learning with the Average Reward Criterioncited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 7; 2026: 7.

15 papers.

Sort:

By area:

  1. 2024How does Inverse RL Scale to Large State Spaces? A Provably Efficient Approachcited by 5
  2. 2025IL-SOAR : Imitation Learning with Soft Optimistic Actor cRiticcited by 5
  3. 2025Inverse Q-Learning Done Right: Offline Imitation Learning in Qπ-Realizable MDPscited by 5
  4. 2025Efficient Imitation under Misspecificationcited by 4
  5. 2025Learning Equilibria from Data: Provably Efficient Multi-Agent Imitation Learningcited by 4
  6. 2025Rate optimal learning of equilibria from datacited by 2
  7. 2025Near-Optimal Second-Order Guarantees for Model-Based Adversarial Imitation Learningcited by 2
  8. 2025Imitation Learning as Return Distribution Matchingcited by 1
  9. 2026Multi-agent imitation learning with function approximation: Linear Markov games and beyondcited by 1
  10. 2026Adversarial Imitation Learning with General Function Approximation: Theoretical Analysis and Practical Algorithmscited by 1
  11. 2026Provably avoiding over-optimization in Direct Preference Optimization without knowing the data distributioncited by 0
  12. 2026Non-Adversarial Imitation Learning Provably Free of Compounding Errors: The Value Flow Mechanismcited by 0
3 more
  1. 2026Split the Differences, Pool the Rest: Provably Efficient Multi-Objective Imitationcited by 0
  2. 2026Provably Efficient Policy-Reward Co-Pretraining for Adversarial Imitation Learningcited by 0
  3. 2026Provable Benefits of Regularization: Fast Rates for Adversarial Imitation Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021On Generalization of Adversarial Imitation Learning and Beyond
  2. 2025Optimistically Optimistic Exploration for Provably Efficient Infinite-Horizon Reinforcement and Imitation Learning
  3. 2019Provably Efficient Imitation Learning from Observation Alone
  4. 2022Multi-Task Imitation Learning for Linear Dynamical Systems
  5. 2021Provably Breaking the Quadratic Error Compounding Barrier in Imitation Learning, Optimally
  6. 2024Provably Efficient Off-Policy Adversarial Imitation Learning with Convergence Guarantees
  7. 2019On the Global Convergence of Imitation Learning: A Case for Linear Quadratic Regulator
  8. 2024A Primal-Dual Algorithm for Offline Constrained Reinforcement Learning with Linear MDPs