# BaRC: Backward Reachability Curriculum for Robotic Reinforcement Learning

arXiv: https://arxiv.org/abs/1806.06161 · HTML: https://mlsanity.com/paper/1806.06161/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jun 2018.

## Built on by (most cited first, 23 of 23)

- [Jump-Start Reinforcement Learning](https://mlsanity.com/paper/2204.02372/) (2022)
- [Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problems](https://mlsanity.com/paper/2111.04613/) (2021)
- [CurricuLLM: Automatic Task Curricula Design for Learning Complex Robot Skills using Large Language Models](https://mlsanity.com/paper/2409.18382/) (2024)
- [Understanding the Complexity Gains of Single-Task RL with a Curriculum](https://mlsanity.com/paper/2212.12809/) (2022)
- [Hamilton-Jacobi Reachability in Reinforcement Learning: A Survey](https://mlsanity.com/paper/2407.09645/) (2024)
- [Reverse Forward Curriculum Learning for Extreme Sample and Demonstration Efficiency in Reinforcement Learning](https://mlsanity.com/paper/2405.03379/) (2024)
- [Robust Reinforcement Learning via Genetic Curriculum](https://mlsanity.com/paper/2202.08393/) (2022)
- [Emergent Dexterity via Diverse Resets and Large-Scale Reinforcement Learning](https://mlsanity.com/paper/2603.15789/) (2026)
- [Automated Reinforcement Learning: An Overview](https://mlsanity.com/paper/2201.05000/) (2022)
- [Automatic Curricula via Expert Demonstrations](https://mlsanity.com/paper/2106.09159/) (2021)
- [Bridging Model-based Safety and Model-free Reinforcement Learning through System Identification of Low Dimensional Linear Models](https://mlsanity.com/paper/2205.05787/) (2022)
- [Event Tables for Efficient Experience Replay](https://mlsanity.com/paper/2211.00576/) (2022)
- [Reward Training Wheels: Adaptive Auxiliary Rewards for Robotics Reinforcement Learning](https://mlsanity.com/paper/2503.15724/) (2025)
- [Learning Multi-Objective Curricula for Robotic Policy Learning](https://mlsanity.com/paper/2110.03032/) (2021)
- [Using Deep Reinforcement Learning with Automatic Curriculum Learning for Mapless Navigation in Intralogistics](https://mlsanity.com/paper/2202.11512/) (2022)
- [Evolutionary Curriculum Training for DRL-Based Navigation Systems](https://mlsanity.com/paper/2306.08870/) (2023)
- [A Dual Curriculum Learning Framework for Multi-UAV Pursuit-Evasion in Diverse Environments](https://mlsanity.com/paper/2312.12255/) (2023)
- [A model-based framework for learning transparent swarm behaviors](https://mlsanity.com/paper/2103.05343/) (2021)
- [Quantification before Selection: Active Dynamics Preference for Robust Reinforcement Learning](https://mlsanity.com/paper/2209.11596/) (2022)
- [CUDC: A Curiosity-Driven Unsupervised Data Collection Method with Adaptive Temporal Distances for Offline Reinforcement Learning](https://mlsanity.com/paper/2312.12191/) (2023)
- [MedLoc-R1: Performance-Aware Curriculum Reward Scheduling for GRPO-Based Medical Visual Grounding](https://mlsanity.com/paper/2603.28120/) (2026)
- [NeuralParker: A Reinforcement Learning Planner for Irregular Parking Environments](https://mlsanity.com/paper/2608.24485/) (2026)
- [Bidirectional Voronoi-biased Exploration Curriculum for Reinforcement Learning](https://mlsanity.com/paper/2610.03395/) (2026)

## Abstract (arXiv)

Model-free Reinforcement Learning (RL) offers an attractive approach to learn control policies for high-dimensional systems, but its relatively poor sample complexity often forces training in simulated environments. Even in simulation, goal-directed tasks whose natural reward function is sparse remain intractable for state-of-the-art model-free algorithms for continuous control. The bottleneck in these tasks is the prohibitive amount of exploration required to obtain a learning signal from the initial state of the system. In this work, we leverage physical priors in the form of an approximate system dynamics model to design a curriculum scheme for a model-free policy optimization algorithm. Our Backward Reachability Curriculum (BaRC) begins policy training from states that require a small number of actions to accomplish the task, and expands the initial state distribution backwards in a dynamically-consistent manner once the policy optimization algorithm demonstrates sufficient performance. BaRC is general, in that it can accelerate training of any model-free RL algorithm on a broad class of goal-directed continuous control MDPs. Its curriculum strategy is physically intuitive, easy-to-tune, and allows incorporating physical priors to accelerate training without hindering the performance, flexibility, and applicability of the model-free RL algorithm. We evaluate our approach on two representative dynamic robotic learning problems and find substantial performance improvement relative to previous curriculum generation techniques and naive exploration strategies.