Divide & Conquer Imitation Learning
On the map: Offline RL and imitation learning from data → Imitation learning methods
Abstract (arXiv)
When cast into the Deep Reinforcement Learning framework, many robotics tasks require solving a long horizon and sparse reward problem, where learning algorithms struggle. In such context, Imitation Learning (IL) can be a powerful approach to bootstrap the learning process. However, most IL methods require several expert demonstrations which can be prohibitively difficult to acquire. Only a handful of IL algorithms have shown efficiency in the context of an extreme low expert data regime where a single expert demonstration is available. In this paper, we present a novel algorithm designed to imitate complex robotic tasks from the states of an expert trajectory. Based on a sequential inductive bias, our method divides the complex task into smaller skills. The skills are learned into a goal-conditioned policy that is able to solve each skill individually and chain skills to solve the entire task. We show that our method imitates a non-holonomic navigation task and scales to a complex simulated robotic manipulation task with very high sample efficiency.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 20 of its 38 references that are in the map. First posted on arXiv 15 Apr 2022.
Builds on
20 of 38 references.
Closest ancestors · 3
Same direction, newest first
- 2020Primal Wasserstein Imitation Learning8 Jun
- 2019Goal-conditioned Imitation Learning13 Jun
- 2019Go-Explore: a New Approach for Hard-Exploration Problems30 Jan
Other references · 17
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2016Generative Adversarial Imitation Learningcited by 1,576
- 2017Hindsight Experience Replaycited by 1,096
- 2018Exploration by Random Network Distillationcited by 966
- 2019Solving Rubik's Cube with a Robot Handcited by 699
- 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
- 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
- 2020First return, then explorecited by 256
- 2017Reverse Curriculum Generation for Reinforcement Learningcited by 185
- 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 169
- 2019Planning with Goal-Conditioned Policiescited by 144
- 2018Learning Montezuma's Revenge from a Single Demonstrationcited by 76
5 more
- 2017OptionGAN: Learning Joint Reward-Policy Options using Generative Adversarial Inverse Reinforcement Learningcited by 40
- 2019A Hitchhiker's Guide to Statistical Comparisons of Reinforcement Learning Algorithmscited by 26
- 2019The problem with DDPG: understanding failures in deterministic environments with sparse rewardscited by 20
- 2018Backplay: "Man muss immer umkehren"cited by 16
- 2020PBCS : Efficient Exploration and Exploitation Using a Synergy between Reinforcement Learning and Motion Planningcited by 11
Built on by
Citing papers per year: 2022: 1; 2023: 0; 2024: 1.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning
- 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations
- 2022Imitation Learning by Estimating Expertise of Demonstrators
- 2022Learning and Retrieval from Prior Data for Skill-based Imitation Learning
- 2023Imitation Bootstrapped Reinforcement Learning
- 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges
- 2020Scalable Multi-Task Imitation Learning with Autonomous Improvement
- 2025Imitation Learning in the Deep Learning Era: A Novel Taxonomy and Recent Advances