mlsanityData to 14 Oct 2026

Theoretical Analysis of Offline Imitation With Supplementary Dataset

arXiv 2301.11687 ↗First posted 27 Jan 2023cs.LGcited by 2

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Behavioral cloning (BC) can recover a good policy from abundant expert data, but may fail when expert data is insufficient. This paper considers a situation where, besides the small amount of expert data, a supplementary dataset is available, which can be collected cheaply from sub-optimal policies. Imitation learning with a supplementary dataset is an emergent practical framework, but its theoretical foundation remains under-developed. To advance understanding, we first investigate a direct extension of BC, called NBCU, that learns from the union of all available data. Our analysis shows that, although NBCU suffers an imitation gap that is larger than BC in the worst case, there exist special cases where NBCU performs better than or equally well as BC. This discovery implies that noisy data can also be helpful if utilized elaborately. Therefore, we further introduce a discriminator-based importance sampling technique to re-weight the supplementary data, proposing the WBCU method. With our newly developed landscape-based analysis, we prove that WBCU can outperform BC in mild conditions. Empirical studies show that WBCU simultaneously achieves the best performance on two challenging tasks where prior state-of-the-art methods fail.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 21 of its 39 references that are in the map. First posted on arXiv 27 Jan 2023.

Builds on

21 of 39 references.

Closest ancestors · 11

Same direction, newest first

  1. 2022Understanding Adversarial Imitation Learning in Small Sample Regime: A Stage-coupled Analysis3 Aug
  2. 2022Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrations20 Jul
  3. 2022Rethinking ValueDice: Does It Really Improve Performance?5 Feb
  4. 2022Robust Imitation Learning from Corrupted Demonstrations29 Jan
  5. 2021Feedback in Imitation Learning: The Three Regimes of Covariate Shift4 Feb
  6. 2020Error Bounds of Imitating Policies and Environments22 Oct
  7. 2020Toward the Fundamental Limits of Imitation Learning13 Sep
  8. 2019A Divergence Minimization Perspective on Imitation Learning Methods6 Nov
  9. 2019Imitation Learning from Imperfect Demonstration27 Jan
  10. 2018An Algorithmic Perspective on Imitation Learning16 Nov
  11. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep

Other references · 10

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2020Learning to summarize from human feedbackcited by 2,588
  3. 2021Convex optimizationcited by 2,515
  4. 2017Improved Training of Wasserstein GANscited by 2,235
  5. 2016Generative Adversarial Imitation Learningcited by 1,597
  6. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 641
  7. 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 539
  8. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 260
  9. 2020FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPscited by 233
  10. 2016Error bounds, quadratic growth, and linear convergence of proximal methodscited by 60

Built on by

Cited by year

Citing papers per year: 2023: 2.

2 papers.

  1. 2023Reward-Consistent Dynamics Models are Strongly Generalizable for Offline Reinforcement Learningcited by 12
  2. 2023Imitation Learning from Purified Demonstrationscited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2022Model-based Offline Imitation Learning with Non-expert Data
  2. 2024How to Leverage Diverse Demonstrations in Offline Imitation Learning
  3. 2024Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning
  4. 2021Mitigating Covariate Shift in Imitation Learning via Offline Data Without Great Coverage
  5. 2025Using Non-Expert Data to Robustify Imitation Learning via Offline Reinforcement Learning
  6. 2024Robust Offline Imitation Learning from Diverse Auxiliary Data
  7. 2025Counterfactual Behavior Cloning: Offline Imitation Learning from Imperfect Human Demonstrations
  8. 2023Offline Imitation Learning with Suboptimal Demonstrations via Relaxed Distribution Matching