# DriverGym: Democratising Reinforcement Learning for Autonomous Driving

arXiv: https://arxiv.org/abs/2111.06889 · HTML: https://mlsanity.com/paper/2111.06889/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 11 of its 17 references that are in the map.
- First posted on arXiv 12 Nov 2021.

## Placed in

- End-to-end autonomous driving / [Reinforcement learning for driving](https://mlsanity.com/driving/reinforcement-learning-for-driving/)

## Builds on (most cited first, 11 of 11)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://mlsanity.com/paper/1712.01815/) (2017)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst](https://mlsanity.com/paper/1812.03079/) (2018)
- [One Thousand and One Hours: Self-driving Motion Prediction Dataset](https://mlsanity.com/paper/2006.14480/) (2020)
- [SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving](https://mlsanity.com/paper/2010.09776/) (2020)
- [SimNet: Learning Reactive Self-driving Simulations from Real-world Observations](https://mlsanity.com/paper/2105.12332/) (2021)
- [Variable Impedance Control in End-Effector Space: An Action Space for Reinforcement Learning in Contact-Rich Tasks](https://mlsanity.com/paper/1906.08880/) (2019)
- [CARLA Real Traffic Scenarios -- novel training ground and benchmark for autonomous driving](https://mlsanity.com/paper/2012.11329/) (2020)

## Built on by (most cited first, 23 of 23)

- [Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research](https://mlsanity.com/paper/2310.08710/) (2023)
- [Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios](https://mlsanity.com/paper/2212.11419/) (2022)
- [The Waymo Open Sim Agents Challenge](https://mlsanity.com/paper/2305.12032/) (2023)
- [Nocturne: a scalable driving benchmark for bringing multi-agent learning one step closer to the real world](https://mlsanity.com/paper/2206.09889/) (2022)
- [BehaviorGPT: Smart Agent Simulation for Autonomous Driving with Next-Patch Prediction](https://mlsanity.com/paper/2405.17372/) (2024)
- [GPUDrive: Data-driven, multi-agent driving simulation at 1 million FPS](https://mlsanity.com/paper/2408.01584/) (2024)
- [Data-driven Traffic Simulation: A Comprehensive Review](https://mlsanity.com/paper/2310.15975/) (2023)
- [Sample Efficient Deep Reinforcement Learning via Uncertainty Estimation](https://mlsanity.com/paper/2201.01666/) (2022)
- [Choose Your Simulator Wisely: A Review on Open-source Simulators for Autonomous Driving](https://mlsanity.com/paper/2311.11056/) (2023)
- [MetaUrban: An Embodied AI Simulation Platform for Urban Micromobility](https://mlsanity.com/paper/2407.08725/) (2024)
- [Towards Autonomous Micromobility through Scalable Urban Simulation](https://mlsanity.com/paper/2505.00690/) (2025)
- [BridgeSim: Unveiling the OL-CL Gap in End-to-End Autonomous Driving](https://mlsanity.com/paper/2604.10856/) (2026)
- [RITA: Boost Driving Simulators with Realistic Interactive Traffic Flow](https://mlsanity.com/paper/2211.03408/) (2022)
- [FusionPlanner: A Multi-task Motion Planner for Mining Trucks via Multi-sensor Fusion](https://mlsanity.com/paper/2308.06931/) (2023)
- [UrbanVerse: Scaling Urban Simulation by Watching City-Tour Videos](https://mlsanity.com/paper/2510.15018/) (2025)
- [Adversarial and Reactive Traffic Entities for Behavior-Realistic Driving Simulation: A Review](https://mlsanity.com/paper/2409.14196/) (2024)
- [ApolloRL: a Reinforcement Learning Platform for Autonomous Driving](https://mlsanity.com/paper/2201.12609/) (2022)
- [GPD-1: Generative Pre-training for Driving](https://mlsanity.com/paper/2412.08643/) (2024)
- [Query The Agent: Improving sample efficiency through epistemic uncertainty estimation](https://mlsanity.com/paper/2210.02585/) (2022)
- [Hybrid Imitation-Learning Motion Planner for Urban Driving](https://mlsanity.com/paper/2409.02871/) (2024)
- [TreeIRL: Safe Urban Driving with Tree Search and Inverse Reinforcement Learning](https://mlsanity.com/paper/2509.13579/) (2025)
- [MILER: Semantic Mid-Level Representation for Sim-to-Real Reinforcement Learning in Unstructured Autonomous Driving](https://mlsanity.com/paper/2609.20747/) (2026)
- [DreamStream: Towards Policy-Oriented Generative Simulation for End-to-End Driving](https://mlsanity.com/paper/2609.26792/) (2026)

## Abstract (arXiv)

Despite promising progress in reinforcement learning (RL), developing algorithms for autonomous driving (AD) remains challenging: one of the critical issues being the absence of an open-source platform capable of training and effectively validating the RL policies on real-world data. We propose DriverGym, an open-source OpenAI Gym-compatible environment specifically tailored for developing RL algorithms for autonomous driving. DriverGym provides access to more than 1000 hours of expert logged data and also supports reactive and data-driven agent behavior. The performance of an RL policy can be easily validated on real-world data using our extensive and flexible closed-loop evaluation protocol. In this work, we also provide behavior cloning baselines using supervised learning and RL, trained in DriverGym. We make DriverGym code, as well as all the baselines publicly available to further stimulate development from the community.