# Formula RL: Deep Reinforcement Learning for Autonomous Racing using Telemetry Data

arXiv: https://arxiv.org/abs/2104.11106 · HTML: https://mlsanity.com/paper/2104.11106/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 24 references that are in the map.
- First posted on arXiv 22 Apr 2021.

## Builds on (most cited first, 11 of 11)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards](https://mlsanity.com/paper/1707.08817/) (2017)
- [DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving](https://mlsanity.com/paper/1505.00256/) (2015)
- [End-to-End Race Driving with Deep Reinforcement Learning](https://mlsanity.com/paper/1807.02371/) (2018)
- [Simulated Car Racing Championship: Competition Software Manual](https://mlsanity.com/paper/1304.1672/) (2013)
- [Meta learning Framework for Automated Driving](https://mlsanity.com/paper/1706.04038/) (2017)
- [Aggressive Deep Driving: Model Predictive Control with a CNN Cost Model](https://mlsanity.com/paper/1707.05303/) (2017)

## Built on by (most cited first, 10 of 10)

- [Autonomous Vehicles on the Edge: A Survey on Autonomous Vehicle Racing](https://mlsanity.com/paper/2202.07008/) (2022)
- [A Super-human Vision-based Reinforcement Learning Agent for Autonomous Racing in Gran Turismo](https://mlsanity.com/paper/2406.12563/) (2024)
- [A Champion-level Vision-based Reinforcement Learning Agent for Competitive Racing in Gran Turismo 7](https://mlsanity.com/paper/2504.09021/) (2025)
- [Automated Reward Design for Gran Turismo](https://mlsanity.com/paper/2511.02094/) (2025)
- [Behavioural Cloning in VizDoom](https://mlsanity.com/paper/2401.03993/) (2024)
- [AARK: An Open Toolkit for Autonomous Racing Research](https://mlsanity.com/paper/2410.00358/) (2024)
- [Racing Towards Reinforcement Learning based control of an Autonomous Formula SAE Car](https://mlsanity.com/paper/2308.13088/) (2023)
- [Learning Autonomous Race Driving with Action Mapping Reinforcement Learning](https://mlsanity.com/paper/2406.14934/) (2024)
- [Vision based driving agent for race car simulation environments](https://mlsanity.com/paper/2504.10266/) (2025)
- [Spatial Envelope MPC: High Performance Driving without a Reference](https://mlsanity.com/paper/2509.18506/) (2025)

## Abstract (arXiv)

This paper explores the use of reinforcement learning (RL) models for autonomous racing. In contrast to passenger cars, where safety is the top priority, a racing car aims to minimize the lap-time. We frame the problem as a reinforcement learning task with a multidimensional input consisting of the vehicle telemetry, and a continuous action space. To find out which RL methods better solve the problem and whether the obtained models generalize to driving on unknown tracks, we put 10 variants of deep deterministic policy gradient (DDPG) to race in two experiments: i)~studying how RL methods learn to drive a racing car and ii)~studying how the learning scenario influences the capability of the models to generalize. Our studies show that models trained with RL are not only able to drive faster than the baseline open source handcrafted bots but also generalize to unknown tracks.