mlsanityData to 14 Oct 2026

Demonstration-free Autonomous Reinforcement Learning via Implicit and Bidirectional Curriculum

arXiv 2305.09943 ↗First posted 17 May 2023cs.LG cs.AI cs.ROcited by 3

Not on the map. Why?

Abstract (arXiv)

While reinforcement learning (RL) has achieved great success in acquiring complex skills solely from environmental interactions, it assumes that resets to the initial state are readily available at the end of each episode. Such an assumption hinders the autonomous learning of embodied agents due to the time-consuming and cumbersome workarounds for resetting in the physical world. Hence, there has been a growing interest in autonomous RL (ARL) methods that are capable of learning from non-episodic interactions. However, existing works on ARL are limited by their reliance on prior data and are unable to learn in environments where task-relevant interactions are sparse. In contrast, we propose a demonstration-free ARL algorithm via Implicit and Bi-directional Curriculum (IBC). With an auxiliary agent that is conditionally activated upon learning progress and a bidirectional goal curriculum based on optimal transport, our method outperforms previous methods, even the ones that leverage demonstrations.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 41 of its 49 references that are in the map. First posted on arXiv 17 May 2023.

Builds on

41 of 49 references.

References · 41

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2016OpenAI Gymcited by 1,736
  3. 2017Hindsight Experience Replaycited by 1,136
  4. 2018Exploration by Random Network Distillationcited by 993
  5. 2018Learning Dexterous In-Hand Manipulationcited by 687
  6. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  7. 2019Emergent Tool Use From Multi-Agent Autocurriculacited by 313
  8. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 260
  9. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
  10. 2019Deep Dynamics Models for Learning Dexterous Manipulationcited by 206
  11. 2020Prioritized Level Replaycited by 198
  12. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 175
29 more
  1. 2017Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Playcited by 141
  2. 2018Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guaranteescited by 140
  3. 2019TossingBot: Learning to Throw Arbitrary Objects with Residual Physicscited by 137
  4. 2019Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environmentscited by 119
  5. 2020The Ingredients of Real-World Robotic Reinforcement Learningcited by 117
  6. 2020Learning to Walk in the Real World with Minimal Human Effortcited by 111
  7. 2019AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videoscited by 105
  8. 2020C-Learning: Learning to Achieve Goals via Recursive Classificationcited by 103
  9. 2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 95
  10. 2018Lipschitz Continuity in Model-based Reinforcement Learningcited by 91
  11. 2017Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement Learningcited by 90
  12. 2018Variational Inverse Control with Events: A General Framework for Data-Driven Reward Definitioncited by 69
  13. 2019Exploration via Hindsight Goal Generationcited by 54
  14. 2020Reset-Free Lifelong Learning with Skill-Space Planningcited by 38
  15. 2021Replacing Rewards with Examples: Example-Based Policy Search via Recursive Classificationcited by 38
  16. 2021Autonomous Reinforcement Learning via Subgoal Curriculacited by 33
  17. 2021Autonomous Reinforcement Learning: Formalism and Benchmarkingcited by 33
  18. 2020Emergent Real-World Robotic Skills via Unsupervised Off-Policy Reinforcement Learningcited by 31
  19. 2020Continual Learning of Control Primitives: Skill Discovery via Reset-Gamescited by 29
  20. 2022Don't Start From Scratch: Leveraging Prior Data to Automate Robotic Reinforcement Learningcited by 29
  21. 2022Curriculum Reinforcement Learning using Optimal Transport via Gradual Domain Adaptationcited by 28
  22. 2016Collective Robot Reinforcement Learning with Distributed Asynchronous Guided Policy Searchcited by 25
  23. 2022You Only Live Once: Single-Life Reinforcement Learningcited by 24
  24. 2022Dexterous Manipulation from Images: Autonomous Real-World RL via Substep Guidancecited by 23
  25. 2022A State-Distribution Matching Approach to Non-Episodic Reinforcement Learningcited by 22
  26. 2022When to Ask for Help: Proactive Interventions in Autonomous Reinforcement Learningcited by 21
  27. 2023Outcome-directed Reinforcement Learning by Uncertainty & Temporal Distance-Aware Curriculum Goal Generationcited by 15
  28. 2020Ecological Reinforcement Learningcited by 13
  29. 2022Automating Reinforcement Learning with Example-based Resetscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 0; 2026: 1.

3 papers.

  1. 2024Intelligent Switching for Reset-Free RLcited by 4
  2. 2024Reset-free Reinforcement Learning with World Modelscited by 3
  3. 2026Bidirectional Voronoi-biased Exploration Curriculum for Reinforcement Learningcited by 0