# Automatic Curriculum Learning for Driving Scenarios: Towards Robust and Efficient Reinforcement Learning

arXiv: https://arxiv.org/abs/2505.08264 · HTML: https://mlsanity.com/paper/2505.08264/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 30 of its 48 references that are in the map.
- First posted on arXiv 13 May 2025.

## Placed in

- End-to-end autonomous driving / [Reinforcement learning for driving](https://mlsanity.com/driving/reinforcement-learning-for-driving/)
- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 30 of 30)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [Learning Lane Graph Representations for Motion Forecasting](https://mlsanity.com/paper/2007.13732/) (2020)
- [Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design](https://mlsanity.com/paper/2012.02096/) (2020)
- [Teacher-Student Curriculum Learning](https://mlsanity.com/paper/1707.00183/) (2017)
- [Prioritized Level Replay](https://mlsanity.com/paper/2010.03934/) (2020)
- [TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors](https://mlsanity.com/paper/2101.06557/) (2021)
- [Evolving Curricula with Regret-Based Environment Design](https://mlsanity.com/paper/2203.01302/) (2022)
- [AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehicles](https://mlsanity.com/paper/2101.06549/) (2021)
- [Self-Supervised Policy Adaptation during Deployment](https://mlsanity.com/paper/2007.04309/) (2020)
- [Replay-Guided Adversarial Environment Design](https://mlsanity.com/paper/2110.02439/) (2021)
- [TrafficGen: Learning to Generate Diverse and Realistic Traffic Scenarios](https://mlsanity.com/paper/2210.06609/) (2022)
- [Interpretable End-to-end Urban Autonomous Driving with Latent Deep Reinforcement Learning](https://mlsanity.com/paper/2001.08726/) (2020)
- [Model-free Deep Reinforcement Learning for Urban Autonomous Driving](https://mlsanity.com/paper/1904.09503/) (2019)
- [SceneGen: Learning to Generate Realistic Traffic Scenes](https://mlsanity.com/paper/2101.06541/) (2021)
- [GoRela: Go Relative for Viewpoint-Invariant Motion Forecasting](https://mlsanity.com/paper/2211.02545/) (2022)
- [Adversarial Reinforcement Learning for Procedural Content Generation](https://mlsanity.com/paper/2103.04847/) (2021)
- [Trajectory Planning for Autonomous Vehicles Using Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2011.04752/) (2020)
- [Transferring Multi-Agent Reinforcement Learning Policies for Autonomous Driving using Sim-to-Real](https://mlsanity.com/paper/2203.11653/) (2022)
- [State Dropout-Based Curriculum Reinforcement Learning for Self-Driving at Unsignalized Intersections](https://mlsanity.com/paper/2207.04361/) (2022)
- [Analysis of Randomization Effects on Sim2Real Transfer in Reinforcement Learning for Robotic Manipulation Tasks](https://mlsanity.com/paper/2206.06282/) (2022)
- [Adversarial Domain Randomization](https://mlsanity.com/paper/1812.00491/) (2018)
- [An End-to-end Deep Reinforcement Learning Approach for the Long-term Short-term Planning on the Frenet Space](https://mlsanity.com/paper/2011.13098/) (2020)
- [KI-PMF: Knowledge Integrated Plausible Motion Forecasting](https://mlsanity.com/paper/2310.12007/) (2023)
- [Informed Reinforcement Learning for Situation-Aware Traffic Rule Exceptions](https://mlsanity.com/paper/2402.04168/) (2024)
- [End-to-end Lidar-Driven Reinforcement Learning for Autonomous Racing](https://mlsanity.com/paper/2309.00296/) (2023)

## Built on by (most cited first, 3 of 3)

- [Scaling Curriculum Learning For Autonomous Driving](https://mlsanity.com/paper/2608.22549/) (2026)
- [From Attacks to Curricula: Learnability-Guided Adversarial Training for Safe Autonomous Driving](https://mlsanity.com/paper/2606.14032/) (2026)
- [Lamarck's Driving School: Discovering Autonomous Driving Training Strategies through Evolutionary Competition](https://mlsanity.com/paper/2610.11662/) (2026)

## Abstract (arXiv)

This paper addresses the challenges of training end-to-end autonomous driving agents using Reinforcement Learning (RL). RL agents are typically trained in a fixed set of scenarios and nominal behavior of surrounding road users in simulations, limiting their generalization and real-life deployment. While domain randomization offers a potential solution by randomly sampling driving scenarios, it frequently results in inefficient training and sub-optimal policies due to the high variance among training scenarios. To address these limitations, we propose an automatic curriculum learning framework that dynamically generates driving scenarios with adaptive complexity based on the agent's evolving capabilities. Unlike manually designed curricula that introduce expert bias and lack scalability, our framework incorporates a ``teacher'' that automatically generates and mutates driving scenarios based on their learning potential -- an agent-centric metric derived from the agent's current policy -- eliminating the need for expert design. The framework enhances training efficiency by excluding scenarios the agent has mastered or finds too challenging. We evaluate our framework in a reinforcement learning setting where the agent learns a driving policy from camera images. Comparative results against baseline methods, including fixed scenario training and domain randomization, demonstrate that our approach leads to enhanced generalization, achieving higher success rates: +9% in low traffic density, +21% in high traffic density, and faster convergence with fewer training steps. Our findings highlight the potential of ACL in improving the robustness and efficiency of RL-based autonomous driving agents.