mlsanityData to 14 Oct 2026

Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement Learning

arXiv 1711.06782 ↗First posted 18 Nov 2017cs.LG cs.ROcited by 90

Not on the map. Why?

Cited by 21 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Deep reinforcement learning algorithms can learn complex behavioral skills, but real-world application of these methods requires a large amount of experience to be collected by the agent. In practical settings, such as robotics, this involves repeatedly attempting a task, resetting the environment between each attempt. However, not all tasks are easily or automatically reversible. In practice, this learning process requires extensive human intervention. In this work, we propose an autonomous method for safe and efficient reinforcement learning that simultaneously learns a forward and reset policy, with the reset policy resetting the environment for a subsequent attempt. By learning a value function for the reset policy, we can automatically determine when the forward policy is about to enter a non-reversible state, providing for uncertainty-aware safety aborts. Our experiments illustrate that proper use of the reset policy can greatly reduce the number of manual resets required to learn a task, can reduce the number of unsafe actions that lead to non-reversible states, and can automatically induce a curriculum.

Key facts. Cited by 90 papers in the map (data through 14 Oct 2026). First posted on arXiv 18 Nov 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 19; 2022: 18; 2023: 16; 2024: 9; 2025: 10; 2026: 17.

90 papers.

Sort:

By area:

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  2. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 213
  3. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 176
  4. 2024SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learningcited by 136
  5. 2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 95
  6. 2021Believe What You See: Implicit Constraint Approach for Offline Multi-Agent Reinforcement Learningcited by 79
  7. 2022Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentationcited by 69
  8. 2022Safe Reinforcement Learning by Imagining the Near Futurecited by 67
  9. 2025RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learningcited by 59
  10. 2022Constrained Policy Optimization via Bayesian World Modelscited by 57
  11. 2023Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learningcited by 54
  12. 2021Safe Reinforcement Learning Using Advantage-Based Interventioncited by 47
38 more
  1. 2021Learning Barrier Certificates: Towards Safe Reinforcement Learning with Zero Training-time Violationscited by 39
  2. 2021Fully Autonomous Real-World Reinforcement Learning with Applications to Mobile Manipulationcited by 35
  3. 2023Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learningcited by 34
  4. 2021Autonomous Reinforcement Learning via Subgoal Curriculacited by 33
  5. 2021Autonomous Reinforcement Learning: Formalism and Benchmarkingcited by 33
  6. 2022Don't Start From Scratch: Leveraging Prior Data to Automate Robotic Reinforcement Learningcited by 29
  7. 2023Safe Exploration in Reinforcement Learning: A Generalized Formulation and Algorithmscited by 28
  8. 2022Sim-to-Lab-to-Real: Safe Reinforcement Learning with Shielding and Generalization Guaranteescited by 27
  9. 2022You Only Live Once: Single-Life Reinforcement Learningcited by 24
  10. 2022Dexterous Manipulation from Images: Autonomous Real-World RL via Substep Guidancecited by 23
  11. 2022Asynchronous Reinforcement Learning for Real-Time Control of Physical Robotscited by 22
  12. 2022A State-Distribution Matching Approach to Non-Episodic Reinforcement Learningcited by 22
  13. 2022When to Ask for Help: Proactive Interventions in Autonomous Reinforcement Learningcited by 21
  14. 2021Dealing with the Unknown: Pessimistic Offline Reinforcement Learningcited by 19
  15. 2021C-Planning: An Automatic Curriculum for Learning Goal-Reaching Taskscited by 17
  16. 2022Transferring Multi-Agent Reinforcement Learning Policies for Autonomous Driving using Sim-to-Realcited by 17
  17. 2023FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicingcited by 17
  18. 2023REBOOT: Reuse Data for Bootstrapping Efficient Real-World Dexterous Manipulationcited by 15
  19. 2023Autonomous Robotic Reinforcement Learning with Asynchronous Human Feedbackcited by 13
  20. 2022Automating Reinforcement Learning with Example-based Resetscited by 12
  21. 2026Failure-Aware RL: Reliable Offline-to-Online Reinforcement Learning with Self-Recovery for Real-World Manipulationcited by 11
  22. 2022Automated Reinforcement Learning: An Overviewcited by 9
  23. 2022LEADER: Learning Attention over Driving Behaviors for Planning under Uncertaintycited by 9
  24. 2025Efficient Action-Constrained Reinforcement Learning via Acceptance-Rejection Method and Augmented MDPscited by 9
  25. 2021MESA: Offline Meta-RL for Safe Adaptation and Fault Tolerancecited by 8
  26. 2021Improving Safety in Deep Reinforcement Learning using Unsupervised Action Planningcited by 7
  27. 2022Goal-Aware Generative Adversarial Imitation Learning from Imperfect Demonstration for Robotic Cloth Manipulationcited by 7
  28. 2021Explicit Explore, Exploit, or Escape (E4): near-optimal safety-constrained reinforcement learning in polynomial timecited by 6
  29. 2022Effects of Safety State Augmentation on Safe Explorationcited by 6
  30. 2025HASARD: A Benchmark for Vision-Based Safe Reinforcement Learning in Embodied Agentscited by 6
  31. 2026Safe Exploration via Policy Priorscited by 6
  32. 2026UniIntervene: Agentic Intervention for Efficient Real-World Reinforcement Learningcited by 6
  33. 2021Automatic Curricula via Expert Demonstrationscited by 5
  34. 2022Safe Reinforcement Learning using Data-Driven Predictive Controlcited by 5
  35. 2023Evaluating Continual Learning on a Home Robotcited by 5
  36. 2023Extreme Risk Mitigation in Reinforcement Learning using Extreme Value Theorycited by 5
  37. 2024Improving Policy Optimization via varepsilon-Retraincited by 5
  38. 2021DESTA: A Framework for Safe Reinforcement Learning with Markov Games of Interventioncited by 4

50 most cited of 90 All 90 (JSON)