# Learning from Simulation, Racing in Reality

arXiv: https://arxiv.org/abs/2011.13332 · HTML: https://mlsanity.com/paper/2011.13332/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Nov 2020.

## Built on by (most cited first, 16 of 16)

- [Autonomous Vehicles on the Edge: A Survey on Autonomous Vehicle Racing](https://mlsanity.com/paper/2202.07008/) (2022)
- [Safe Autonomous Racing via Approximate Reachability on Ego-vision](https://mlsanity.com/paper/2110.07699/) (2021)
- [A Benchmark Comparison of Imitation Learning-based Control Policies for Autonomous Racing](https://mlsanity.com/paper/2209.15073/) (2022)
- [TC-Driver: Trajectory Conditioned Driving for Robust Autonomous Racing -- A Reinforcement Learning Approach](https://mlsanity.com/paper/2205.09370/) (2022)
- [Autonomous Driving Small-Scale Cars: A Survey of Recent Development](https://mlsanity.com/paper/2404.06229/) (2024)
- [Learn-to-Race Challenge 2022: Benchmarking Safe Learning and Cross-domain Generalisation in Autonomous Racing](https://mlsanity.com/paper/2205.02953/) (2022)
- [Lane-Keeping Control of Autonomous Vehicles Through a Soft-Constrained Iterative LQR](https://mlsanity.com/paper/2311.16900/) (2023)
- [Data-efficient Deep Reinforcement Learning for Vehicle Trajectory Control](https://mlsanity.com/paper/2311.18393/) (2023)
- [RLPP: A Residual Method for Zero-Shot Real-World Autonomous Racing on Scaled Platforms](https://mlsanity.com/paper/2501.17311/) (2025)
- [Drive Fast, Learn Faster: On-Board RL for High Performance Autonomous Racing](https://mlsanity.com/paper/2505.07321/) (2025)
- [Partial End-to-end Reinforcement Learning for Robustness Against Modelling Error in Autonomous Racing](https://mlsanity.com/paper/2312.06406/) (2023)
- [Mixed-Reality Digital Twins: Leveraging the Physical and Virtual Worlds for Hybrid Sim2Real Transition of Multi-Agent Reinforcement Learning Policies](https://mlsanity.com/paper/2403.10996/) (2024)
- [On learning racing policies with reinforcement learning](https://mlsanity.com/paper/2504.02420/) (2025)
- [Physics-Informed Reinforcement Learning of Spatial Density Velocity Potentials for Map-Free Racing](https://mlsanity.com/paper/2604.09499/) (2026)
- [Autonomous overtaking trajectory optimization using reinforcement learning and opponent pose estimation](https://mlsanity.com/paper/2603.27207/) (2026)
- [DAMPER: Return-Prioritized Gradient Control for Smooth Policies](https://mlsanity.com/paper/2609.38903/) (2026)

## Abstract (arXiv)

We present a reinforcement learning-based solution to autonomously race on a miniature race car platform. We show that a policy that is trained purely in simulation using a relatively simple vehicle model, including model randomization, can be successfully transferred to the real robotic setup. We achieve this by using novel policy output regularization approach and a lifted action space which enables smooth actions but still aggressive race car driving. We show that this regularized policy does outperform the Soft Actor Critic (SAC) baseline method, both in simulation and on the real car, but it is still outperformed by a Model Predictive Controller (MPC) state of the art method. The refinement of the policy with three hours of real-world interaction data allows the reinforcement learning policy to achieve lap times similar to the MPC controller while reducing track constraint violations by 50%.