mlsanityData to 7 Oct 2026

Automating Reinforcement Learning with Example-based Resets

arXiv 2204.02041 ↗First posted 5 Apr 2022cs.LG cs.ROcited by 11

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning has enabled robots to learn motor skills from environmental interactions with minimal to no prior knowledge. However, existing reinforcement learning algorithms assume an episodic setting, in which the agent resets to a fixed initial state distribution at the end of each episode, to successfully train the agents from repeated trials. Such reset mechanism, while trivial for simulated tasks, can be challenging to provide for real-world robotics tasks. Resets in robotic systems often require extensive human supervision and task-specific workarounds, which contradicts the goal of autonomous robot learning. In this paper, we propose an extension to conventional reinforcement learning towards greater autonomy by introducing an additional agent that learns to reset in a self-supervised manner. The reset agent preemptively triggers a reset to prevent manual resets and implicitly imposes a curriculum for the forward agent. We apply our method to learn from scratch on a suite of simulated and real-world continuous control tasks and demonstrate that the reset agent successfully learns to reduce manual resets whilst also allowing the forward policy to improve gradually over time.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 27 of its 30 references that are in the map. First posted on arXiv 5 Apr 2022.

Builds on

27 of 30 references.

References · 27

  1. 2015Continuous control with deep reinforcement learningcited by 3,279
  2. 2016OpenAI Gymcited by 1,684
  3. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  4. 2015Prioritized Experience Replaycited by 1,070
  5. 2018DeepMind Control Suitecited by 935
  6. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  7. 2018Learning Dexterous In-Hand Manipulationcited by 643
  8. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 361
  9. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 342
  10. 2018Randomized Prior Functions for Deep Reinforcement Learningcited by 218
  11. 2017Reverse Curriculum Generation for Reinforcement Learningcited by 185
  12. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 156
15 more
  1. 2019TossingBot: Learning to Throw Arbitrary Objects with Residual Physicscited by 131
  2. 2020The Ingredients of Real-World Robotic Reinforcement Learningcited by 111
  3. 2019AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videoscited by 105
  4. 2020Learning to Walk in the Real World with Minimal Human Effortcited by 103
  5. 2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 90
  6. 2017Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement Learningcited by 86
  7. 2018Variational Inverse Control with Events: A General Framework for Data-Driven Reward Definitioncited by 66
  8. 2020Rewriting History with Inverse RL: Hindsight Inference for Policy Improvementcited by 59
  9. 2019Form2Fit: Learning Shape Priors for Generalizable Assembly from Disassemblycited by 51
  10. 2021Replacing Rewards with Examples: Example-Based Policy Search via Recursive Classificationcited by 38
  11. 2021Autonomous Reinforcement Learning via Subgoal Curriculacited by 32
  12. 2020Emergent Real-World Robotic Skills via Unsupervised Off-Policy Reinforcement Learningcited by 30
  13. 2020Continual Learning of Control Primitives: Skill Discovery via Reset-Gamescited by 29
  14. 2016Collective Robot Reinforcement Learning with Distributed Asynchronous Guided Policy Searchcited by 25
  15. 2021There Is No Turning Back: A Self-Supervised Approach for Reversibility-Aware Reinforcement Learningcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 4; 2025: 2; 2026: 1.

11 papers.

  1. 2024Behavior Generation with Latent Actionscited by 159
  2. 2023Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learningcited by 29
  3. 2022You Only Live Once: Single-Life Reinforcement Learningcited by 23
  4. 2025IMLE Policy: Fast and Sample Efficient Visuomotor Policy Learning via Implicit Maximum Likelihood Estimationcited by 13
  5. 2024Intelligent Switching for Reset-Free RLcited by 4
  6. 2026UniIntervene: Agentic Intervention for Efficient Real-World Reinforcement Learningcited by 4
  7. 2024Safety-Driven Deep Reinforcement Learning Framework for Cobots: A Sim2Real Approachcited by 3
  8. 2024Reset-free Reinforcement Learning with World Modelscited by 3
  9. 2023Demonstration-free Autonomous Reinforcement Learning via Implicit and Bidirectional Curriculumcited by 2
  10. 2023Self-Supervised Curriculum Generation for Autonomous Reinforcement Learning without Task-Specific Knowledgecited by 1
  11. 2025Learning Without Time-Based Embodiment Resets in Soft-Actor Criticcited by 1