Online Adaptation for Enhancing Imitation Learning Policies
On the map: Offline RL and imitation learning from data → Imitation learning methods
Abstract (arXiv)
Imitation learning enables autonomous agents to learn from human examples, without the need for a reward signal. Still, if the provided dataset does not encapsulate the task correctly, or when the task is too complex to be modeled, such agents fail to reproduce the expert policy. We propose to recover from these failures through online adaptation. Our approach combines the action proposal coming from a pre-trained policy with relevant experience recorded by an expert. The combination results in an adapted action that closely follows the expert. Our experiments show that an adapted agent performs better than its pure imitation learning counterpart. Notably, adapted agents can achieve reasonable performance even when the base, non-adapted policy catastrophically fails.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 24 of its 36 references that are in the map. First posted on arXiv 7 Jun 2024.
Builds on
24 of 36 references.
Closest ancestors · 4
Same direction, newest first
- 2024Zero-shot Imitation Policy via Search in Demonstration Dataset29 Jan
- 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos23 Jun
- 2021Imitation Learning: Progress, Taxonomies and Challenges23 Jun
- 2020Benchmarking End-to-End Behavioural Cloning on Video Games2 Apr
Other references · 20
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2022Generative Adversarial Networkscited by 5,603
- 2017Deep reinforcement learning from human preferencescited by 3,846
- 2017Billion-scale similarity search with GPUscited by 2,477
- 2016Generative Adversarial Imitation Learningcited by 1,576
- 2023Mastering Diverse Domains through World Modelscited by 1,193
- 2020Mastering Atari with Discrete World Modelscited by 990
- 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
- 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
- 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
8 more
- 2017A Brief Survey of Deep Reinforcement Learningcited by 466
- 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
- 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 320
- 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progresscited by 294
- 2021The Surprising Effectiveness of Representation Learning for Visual Imitationcited by 149
- 2023ChatGPT is not all you need. A State of the Art Review of large Generative AI modelscited by 85
- 2022MineRL Diamond 2021 Competition: Overview, Results, and Lessons Learnedcited by 26
- 2023BEDD: The MineRL BASALT Evaluation and Demonstrations Dataset for Training and Benchmarking Agents that Solve Fuzzy Taskscited by 8
Built on by
Citing papers per year: 2025: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2020Explaining Fast Improvement in Online Imitation Learning
- 2021Imitation by Predicting Observations
- 2024Policy Decorator: Model-Agnostic Online Refinement for Large Policy Model
- 2021Imitation Learning by Reinforcement Learning
- 2024Reward-free World Models for Online Imitation Learning
- 2022Imitation Learning by Estimating Expertise of Demonstrators
- 2025Using Non-Expert Data to Robustify Imitation Learning via Offline Reinforcement Learning
- 2020Scalable Multi-Task Imitation Learning with Autonomous Improvement