# Scaling Is All You Need: Autonomous Driving with JAX-Accelerated Reinforcement Learning

arXiv: https://arxiv.org/abs/2312.15122 · HTML: https://mlsanity.com/paper/2312.15122/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 10 of its 17 references that are in the map.
- First posted on arXiv 23 Dec 2023.

## Placed in

- End-to-end autonomous driving / [Reinforcement learning for driving](https://mlsanity.com/driving/reinforcement-learning-for-driving/)

## Builds on (most cited first, 10 of 10)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures](https://mlsanity.com/paper/1802.01561/) (2018)
- [End-to-End Urban Driving by Imitating a Reinforcement Learning Coach](https://mlsanity.com/paper/2108.08265/) (2021)
- [Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios](https://mlsanity.com/paper/2212.11419/) (2022)
- [SafetyNet: Safe planning for real-world self-driving vehicles using machine-learned policies](https://mlsanity.com/paper/2109.13602/) (2021)
- [Scaling laws for single-agent reinforcement learning](https://mlsanity.com/paper/2301.13442/) (2023)
- [Learning Realistic Traffic Agents in Closed-loop](https://mlsanity.com/paper/2311.01394/) (2023)
- [Embedding Synthetic Off-Policy Experience for Autonomous Driving via Zero-Shot Curricula](https://mlsanity.com/paper/2212.01375/) (2022)

## Built on by (most cited first, 2 of 2)

- [Scaling Self-Play for End-to-End Driving](https://mlsanity.com/paper/2606.19641/) (2026)
- [RDAR: Reward-Driven Agent Relevance Estimation for Autonomous Driving](https://mlsanity.com/paper/2509.19789/) (2025)

## Abstract (arXiv)

Reinforcement learning has been demonstrated to outperform even the best humans in complex domains like video games. However, running reinforcement learning experiments on the required scale for autonomous driving is extremely difficult. Building a large scale reinforcement learning system and distributing it across many GPUs is challenging. Gathering experience during training on real world vehicles is prohibitive from a safety and scalability perspective. Therefore, an efficient and realistic driving simulator is required that uses a large amount of data from real-world driving. We bring these capabilities together and conduct large-scale reinforcement learning experiments for autonomous driving. We demonstrate that our policy performance improves with increasing scale. Our best performing policy reduces the failure rate by 64% while improving the rate of driving progress by 25% compared to the policies produced by state-of-the-art machine learning for autonomous driving.