mlsanityData to 14 Oct 2026

Bidirectional Voronoi-biased Exploration Curriculum for Reinforcement Learning

arXiv 2610.03395 ↗First posted 2 Oct 2026cs.LG cs.ROcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations also fits: RL for robot locomotion and manipulation

Abstract (arXiv)

Long-horizon tasks with sparse rewards pose an exploration bottleneck for goal-conditioned reinforcement learning: a policy started from the initial state rarely reaches the goal and receives no learning signal. Reference motions, hand-designed curricula, and shaped rewards supply this signal but require demonstrations or task-specific engineering; automatic start-state and goal curricula avoid this but typically expand from one side only, so the full distance to the target must be covered from that side. We propose the Bidirectional Voronoi-biased Exploration curriculum for Reinforcement learning (BVER), which expands from both ends at once. Inspired by bidirectional RRT planning, BVER grows start states outward from the goal and goals outward from the initial state distribution, biases both toward unexplored task space, and steers them toward each other, training one goal-conditioned policy on both. On point-mass mazes, quadrupedal box climbing, and robot-arm ring-on-peg transfer, BVER learns faster than all compared reference-free curricula. On box climbing, it reaches 95% success on a 0.4 m box in roughly 65% fewer iterations than the best of them, is the only one of them to learn to climb a 0.7 m box, and yields a policy robust to start, goal, and yaw variation. Without a demonstration, it approaches the sample efficiency of reference-based curricula on the 0.4 m box and on ring-on-peg transfer. Ablations show that expanding from both ends outperforms either direction alone.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 36 of its 45 references that are in the map. First posted on arXiv 2 Oct 2026.

Builds on

36 of 45 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learning26 May

Other references · 35

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Hindsight Experience Replaycited by 1,136
  3. 2018Exploration by Random Network Distillationcited by 993
  4. 2019Solving Rubik's Cube with a Robot Handcited by 734
  5. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  6. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
  7. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  8. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 340
  9. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
  10. 2020First return, then explorecited by 266
  11. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
  12. 2017Reverse Curriculum Generation for Reinforcement Learningcited by 198
23 more
  1. 2020Prioritized Level Replaycited by 198
  2. 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 194
  3. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 174
  4. 2022Evolving Curricula with Regret-Based Environment Designcited by 173
  5. 2023Extreme Parkour with Legged Robotscited by 164
  6. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 158
  7. 2023ANYmal Parkour: Learning Agile Navigation for Quadrupedal Robotscited by 152
  8. 2020Automatic Curriculum Learning For Deep RL: A Short Surveycited by 150
  9. 2019Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environmentscited by 119
  10. 2020Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learningcited by 102
  11. 2018Learning Montezuma's Revenge from a Single Demonstrationcited by 80
  12. 2019Dynamical Distance Learning for Semi-Supervised and Unsupervised Skill Discoverycited by 68
  13. 2019Exploration via Hindsight Goal Generationcited by 54
  14. 2025RSL-RL: A Learning Library for Robotics Researchcited by 49
  15. 2022OPT-Mimic: Imitation of Optimized Trajectories for Dynamic Quadruped Behaviorscited by 35
  16. 2021Autonomous Reinforcement Learning via Subgoal Curriculacited by 33
  17. 2018BaRC: Backward Reachability Curriculum for Robotic Reinforcement Learningcited by 23
  18. 2018Backplay: "Man muss immer umkehren"cited by 21
  19. 2024Curriculum-Based Reinforcement Learning for Quadrupedal Jumping: A Reference-free Designcited by 18
  20. 2021C-Planning: An Automatic Curriculum for Learning Goal-Reaching Taskscited by 17
  21. 2023Outcome-directed Reinforcement Learning by Uncertainty & Temporal Distance-Aware Curriculum Goal Generationcited by 15
  22. 2020PBCS : Efficient Exploration and Exploitation Using a Synergy between Reinforcement Learning and Motion Planningcited by 12
  23. 2023Demonstration-free Autonomous Reinforcement Learning via Implicit and Bidirectional Curriculumcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Proximal Curriculum with Task Correlations for Deep Reinforcement Learning
  2. 2020Guided Exploration with Proximal Policy Optimization using a Single Demonstration
  3. 2025Causal-Paced Deep Reinforcement Learning
  4. 2024Reverse Forward Curriculum Learning for Extreme Sample and Demonstration Efficiency in Reinforcement Learning
  5. 2026Decoupling Task and Behavior: A Two-Stage Reward Curriculum in Reinforcement Learning for Robotics
  6. 2025Test-Time Graph Search for Goal-Conditioned Reinforcement Learning
  7. 2025Exploration Behavior of Untrained Policies
  8. 2021Learning from Guided Play: A Scheduled Hierarchical Approach for Improving Exploration in Adversarial Imitation Learning