mlsanityData to 7 Oct 2026

Leveraging Sequentiality in Reinforcement Learning from a Single Demonstration

arXiv 2211.04786 ↗First posted 9 Nov 2022cs.RO cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Deep Reinforcement Learning has been successfully applied to learn robotic control. However, the corresponding algorithms struggle when applied to problems where the agent is only rewarded after achieving a complex task. In this context, using demonstrations can significantly speed up the learning process, but demonstrations can be costly to acquire. In this paper, we propose to leverage a sequential bias to learn control policies for complex robotic tasks using a single demonstration. To do so, our method learns a goal-conditioned policy to control a system between successive low-dimensional goals. This sequential goal-reaching approach raises a problem of compatibility between successive goals: we need to ensure that the state resulting from reaching a goal is compatible with the achievement of the following goals. To tackle this problem, we present a new algorithm called DCIL-II. We show that DCIL-II can solve with unprecedented sample efficiency some challenging simulated tasks such as humanoid locomotion and stand-up as well as fast running with a simulated Cassie robot. Our method leveraging sequentiality is a step towards the resolution of complex robotic tasks under minimal specification effort, a key feature for the next generation of autonomous robots.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 31 of its 57 references that are in the map. First posted on arXiv 9 Nov 2022.

Builds on

31 of 57 references.

References · 31

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2016OpenAI Gymcited by 1,684
  3. 2016Generative Adversarial Imitation Learningcited by 1,576
  4. 2017Hindsight Experience Replaycited by 1,096
  5. 2018Exploration by Random Network Distillationcited by 966
  6. 2019Solving Rubik's Cube with a Robot Handcited by 699
  7. 2018Learning Dexterous In-Hand Manipulationcited by 643
  8. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  9. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  10. 2016The Option-Critic Architecturecited by 433
  11. 2020First return, then explorecited by 256
  12. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
19 more
  1. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 207
  2. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learningcited by 203
  3. 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 188
  4. 2020Controlling Overestimation Bias with Truncated Mixture of Continuous Distributional Quantile Criticscited by 154
  5. 2018DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skillscited by 153
  6. 2019Planning with Goal-Conditioned Policiescited by 144
  7. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 129
  8. 2020Primal Wasserstein Imitation Learningcited by 120
  9. 2021Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstrationcited by 92
  10. 2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 90
  11. 2018Learning Montezuma's Revenge from a Single Demonstrationcited by 76
  12. 2017When Waiting is not an Option : Learning Options with a Deliberation Costcited by 57
  13. 2018Hierarchical visuomotor control of humanoidscited by 29
  14. 2018Learning from Demonstration in the Wildcited by 25
  15. 2017Learnings Options End-to-End for Continuous Action Taskscited by 24
  16. 2018Backplay: "Man muss immer umkehren"cited by 16
  17. 2020PBCS : Efficient Exploration and Exploitation Using a Synergy between Reinforcement Learning and Motion Planningcited by 11
  18. 2022Learning to Get Upcited by 11
  19. 2022Divide & Conquer Imitation Learningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 2; 2025: 0; 2026: 2.

6 papers.

  1. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 308
  2. 2024SkillMimicGen: Automated Demonstration Generation for Efficient Skill Learning and Deploymentcited by 48
  3. 2023NOD-TAMP: Generalizable Long-Horizon Planning with Neural Object Descriptorscited by 8
  4. 2026Online Self-Training for Co-Adaptation in Hierarchical Diffusion Policiescited by 1
  5. 2024Single-Reset Divide & Conquer Imitation Learningcited by 0
  6. 2026Locally-Guided Actor-Critic: Training a Goal-conditioned Actor with a Subgoal-aware Criticcited by 0