mlsanityData to 7 Oct 2026

Provable Representation Learning for Imitation Learning via Bi-level Optimization

arXiv 2002.10544 ↗First posted 24 Feb 2020cs.LG cs.AI stat.MLcited by 41

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

A common strategy in modern learning systems is to learn a representation that is useful for many tasks, a.k.a. representation learning. We study this strategy in the imitation learning setting for Markov decision processes (MDPs) where multiple experts' trajectories are available. We formulate representation learning as a bi-level optimization problem where the "outer" optimization tries to learn the joint representation and the "inner" optimization encodes the imitation learning setup and tries to learn task-specific parameters. We instantiate this framework for the imitation learning settings of behavior cloning and observation-alone. Theoretically, we show using our framework that representation learning can provide sample complexity benefits for imitation learning in both settings. We also provide proof-of-concept experiments to verify our theory.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 23 of its 43 references that are in the map. First posted on arXiv 24 Feb 2020.

Builds on

23 of 43 references.

Closest ancestors · 4

Same direction, newest first

  1. 2019Provably Efficient Imitation Learning from Observation Alone27 May
  2. 2018Truncated Horizon Policy Search: Combining Reinforcement Learning & Imitation Learning29 May
  3. 2018Imitating Latent Policies from Observation21 May
  4. 2018Behavioral Cloning from Observation4 May

Other references · 19

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
  6. 2019Meta-Learning with Implicit Gradientscited by 523
  7. 2019Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLcited by 359
  8. 2018Meta-Learning: A Surveycited by 254
  9. 2019Batch Policy Learning under Constraintscited by 242
  10. 2017One-Shot Imitation Learningcited by 232
  11. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  12. 2015The Benefit of Multitask Representation Learningcited by 194
7 more
  1. 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
  2. 2014Reinforcement and Imitation Learning via Interactive No-Regret Learningcited by 116
  3. 2018Task-Embedded Control Networks for Few-Shot Imitation Learningcited by 65
  4. 2015Learning to Search Better Than Your Teachercited by 50
  5. 2015Complete Dictionary Recovery over the Sphere I: Overview and the Geometric Picturecited by 49
  6. 2019Learning-to-Learn Stochastic Gradient Descent with Biased Regularizationcited by 49
  7. 2018Incremental Learning-to-Learn with Statistical Guaranteescited by 16

Built on by

Cited by year

Citing papers per year: 2020: 1; 2021: 11; 2022: 10; 2023: 10; 2024: 5; 2025: 4.

41 papers.

Sort:

By area:

  1. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 151
  2. 2020Toward the Fundamental Limits of Imitation Learningcited by 96
  3. 2022FedNest: Federated Bilevel, Minimax, and Compositional Optimizationcited by 60
  4. 2021A Survey of Deep RL and IL for Autonomous Driving Policy Learningcited by 59
  5. 2021Lower Bounds and Accelerated Algorithms for Bilevel Optimizationcited by 46
  6. 2021Near-optimal Representation Learning for Linear Bandits and Linear RLcited by 45
  7. 2021TRAIL: Near-Optimal Imitation Learning with Suboptimal Datacited by 39
  8. 2022Provable Benefits of Representational Transfer in Reinforcement Learningcited by 38
  9. 2021Provable Representation Learning for Imitation with Contrastive Fourier Featurescited by 34
  10. 2022Decentralized Gossip-Based Stochastic Bilevel Optimization over Communication Networkscited by 32
  11. 2021On the Power of Multitask Representation Learning in Linear MDPcited by 30
  12. 2022Chain of Thought Imitation with Procedure Cloningcited by 30
29 more
  1. 2023Inverse Dynamics Pretraining Learns Good Representations for Multitask Imitationcited by 30
  2. 2022Provable Benefit of Multitask Representation Learning in Reinforcement Learningcited by 29
  3. 2022Multi-Task Imitation Learning for Linear Dynamical Systemscited by 22
  4. 2025Language Models Can Learn from Verbal Feedback Without Scalar Rewardscited by 17
  5. 2022Nearly Minimax Algorithms for Linear Bandits with Shared Representationcited by 16
  6. 2021Provably Breaking the Quadratic Error Compounding Barrier in Imitation Learning, Optimallycited by 12
  7. 2022Provable General Function Class Representation Learning in Multitask Bandits and MDPscited by 12
  8. 2022Behavior Prior Representation learning for Offline Reinforcement Learningcited by 12
  9. 2023Aligning Robot and Human Representationscited by 11
  10. 2023Understanding and Addressing the Pitfalls of Bisimulation-based Representations in Offline Reinforcement Learningcited by 10
  11. 2021Learning an Explicit Hyperparameter Prediction Function Conditioned on Taskscited by 9
  12. 2022Min-Max Bilevel Multi-objective Optimization with Applications in Machine Learningcited by 9
  13. 2024Fast and Sample Efficient Multi-Task Representation Learning in Stochastic Contextual Banditscited by 6
  14. 2023A Generalized Alternating Method for Bilevel Learning under the Polyak-Lojasiewicz Conditioncited by 5
  15. 2025IL-SOAR : Imitation Learning with Soft Optimistic Actor cRiticcited by 5
  16. 2021Learning distinct features helps, provablycited by 4
  17. 2021Adaptive Clustering and Personalization in Multi-Agent Stochastic Linear Banditscited by 4
  18. 2024Investigating Pre-Training Objectives for Generalization in Vision-Based Reinforcement Learningcited by 4
  19. 2025Learning Theory for Kernel Bilevel Optimizationcited by 2
  20. 2023A Statistical Guarantee for Representation Transfer in Multitask Imitation Learningcited by 1
  21. 2024The Power of Active Multi-Task Learning in Reinforcement Learning from Human Feedbackcited by 1
  22. 2024Guarantees for Nonlinear Representation Learning: Non-identical Covariates, Dependent Data, Fewer Samplescited by 1
  23. 2025Towards Understanding the Benefit of Multitask Representation Learning in Decision Processcited by 1
  24. 2023Federated Multi-Sequence Stochastic Approximation with Local Hypergradient Estimationcited by 0
  25. 2023BiERL: A Meta Evolutionary Reinforcement Learning Framework via Bilevel Optimizationcited by 0
  26. 2023Imitator Learning: Achieve Out-of-the-Box Imitation Ability in Variable Environmentscited by 0
  27. 2023Federated Multi-Level Optimization over Decentralized Networkscited by 0
  28. 2023Provable Benefits of Multi-task RL under Non-Markovian Decision Making Processescited by 0
  29. 2024Stable Offline Value Function Learning with Bisimulation-based Representationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Difference-Aware Retrieval Policies for Imitation Learning
  2. 2022An Empirical Investigation of Representation Learning for Imitation
  3. 2022Imitation Learning by Estimating Expertise of Demonstrators
  4. 2025A Model-Based Approach to Imitation Learning through Multi-Step Predictions
  5. 2021Imitation Learning: Progress, Taxonomies and Challenges
  6. 2021The Surprising Effectiveness of Representation Learning for Visual Imitation
  7. 2022Learning and Retrieval from Prior Data for Skill-based Imitation Learning
  8. 2024Bayesian Optimization for Sample-Efficient Policy Improvement in Robotic Manipulation ↗