# Autonomous Overtaking in Gran Turismo Sport Using Curriculum Reinforcement Learning

arXiv: https://arxiv.org/abs/2103.14666 · HTML: https://mlsanity.com/paper/2103.14666/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 26 references that are in the map.
- First posted on arXiv 26 Mar 2021.

## Builds on (most cited first, 9 of 9)

- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Off-Policy Deep Reinforcement Learning without Exploration](https://mlsanity.com/paper/1812.02900/) (2018)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehicles](https://mlsanity.com/paper/1604.07446/) (2016)
- [Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement Learning](https://mlsanity.com/paper/2008.07971/) (2020)
- [End-to-End Race Driving with Deep Reinforcement Learning](https://mlsanity.com/paper/1807.02371/) (2018)
- [Deep Latent Competition: Learning to Race Using Visual Control Policies in Latent Space](https://mlsanity.com/paper/2102.09812/) (2021)

## Built on by (most cited first, 30 of 45)

- [Autonomous Vehicles on the Edge: A Survey on Autonomous Vehicle Racing](https://mlsanity.com/paper/2202.07008/) (2022)
- [How Simulation Helps Autonomous Driving:A Survey of Sim2real, Digital Twins, and Parallel Intelligence](https://mlsanity.com/paper/2305.01263/) (2023)
- [CaRL: Learning Scalable Planning Policies with Simple Rewards](https://mlsanity.com/paper/2504.17838/) (2025)
- [Policy Search for Model Predictive Control with Application to Agile Drone Flight](https://mlsanity.com/paper/2112.03850/) (2021)
- [Learning Interactive Driving Policies via Data-driven Simulation](https://mlsanity.com/paper/2111.12137/) (2021)
- [State Dropout-Based Curriculum Reinforcement Learning for Self-Driving at Unsignalized Intersections](https://mlsanity.com/paper/2207.04361/) (2022)
- [Contrastive Learning for Enhancing Robust Scene Transfer in Vision-based Agile Flight](https://mlsanity.com/paper/2309.09865/) (2023)
- [Reward-Driven Automated Curriculum Learning for Interaction-Aware Self-Driving at Unsignalized Intersections](https://mlsanity.com/paper/2403.13674/) (2024)
- [TC-Driver: Trajectory Conditioned Driving for Robust Autonomous Racing -- A Reinforcement Learning Approach](https://mlsanity.com/paper/2205.09370/) (2022)
- [Motion Planning and Control for Multi Vehicle Autonomous Racing at High Speeds](https://mlsanity.com/paper/2207.11136/) (2022)
- [Autonomous Driving at Unsignalized Intersections: A Review of Decision-Making Challenges and Reinforcement Learning-Based Solutions](https://mlsanity.com/paper/2409.13144/) (2024)
- [Knowledge Transfer from Simple to Complex: A Safe and Efficient Reinforcement Learning Framework for Autonomous Driving Decision-Making](https://mlsanity.com/paper/2410.14468/) (2024)
- [BeTAIL: Behavior Transformer Adversarial Imitation Learning from Human Racing Gameplay](https://mlsanity.com/paper/2402.14194/) (2024)
- [Event Tables for Efficient Experience Replay](https://mlsanity.com/paper/2211.00576/) (2022)
- [RLPP: A Residual Method for Zero-Shot Real-World Autonomous Racing on Scaled Platforms](https://mlsanity.com/paper/2501.17311/) (2025)
- [Bilevel Multi-Armed Bandit-Based Hierarchical Reinforcement Learning for Interaction-Aware Self-Driving at Unsignalized Intersections](https://mlsanity.com/paper/2502.03960/) (2025)
- [A Champion-level Vision-based Reinforcement Learning Agent for Competitive Racing in Gran Turismo 7](https://mlsanity.com/paper/2504.09021/) (2025)
- [Expert Human-Level Driving in Gran Turismo Sport Using Deep Reinforcement Learning with Image-based Representation](https://mlsanity.com/paper/2111.06449/) (2021)
- [Tactical Decision Making for Autonomous Trucks by Deep Reinforcement Learning with Total Cost of Operation Based Reward](https://mlsanity.com/paper/2403.06524/) (2024)
- [Automated Reward Design for Gran Turismo](https://mlsanity.com/paper/2511.02094/) (2025)
- [Learning the References of Online Model Predictive Control for Urban Self-Driving](https://mlsanity.com/paper/2308.15808/) (2023)
- [Improving Environment Robustness of Deep Reinforcement Learning Approaches for Autonomous Racing Using Bayesian Optimization-based Curriculum Learning](https://mlsanity.com/paper/2312.10557/) (2023)
- [Trajectory Planning Using Reinforcement Learning for Interactive Overtaking Maneuvers in Autonomous Racing Scenarios](https://mlsanity.com/paper/2404.10658/) (2024)
- [AARK: An Open Toolkit for Autonomous Racing Research](https://mlsanity.com/paper/2410.00358/) (2024)
- [A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspective](https://mlsanity.com/paper/2503.23650/) (2025)
- [Autonomous Racing using a Hybrid Imitation-Reinforcement Learning Architecture](https://mlsanity.com/paper/2110.05437/) (2021)
- [Perception Imitation: Towards Synthesis-free Simulator for Autonomous Vehicles](https://mlsanity.com/paper/2304.09365/) (2023)
- [Sense, Imagine, Act: Multimodal Perception Improves Model-Based Reinforcement Learning for Head-to-Head Autonomous Racing](https://mlsanity.com/paper/2305.04750/) (2023)
- [Composing Efficient, Robust Tests for Policy Selection](https://mlsanity.com/paper/2306.07372/) (2023)
- [Partial End-to-end Reinforcement Learning for Robustness Against Modelling Error in Autonomous Racing](https://mlsanity.com/paper/2312.06406/) (2023)

## Abstract (arXiv)

Professional race-car drivers can execute extreme overtaking maneuvers. However, existing algorithms for autonomous overtaking either rely on simplified assumptions about the vehicle dynamics or try to solve expensive trajectory-optimization problems online. When the vehicle approaches its physical limits, existing model-based controllers struggle to handle highly nonlinear dynamics, and cannot leverage the large volume of data generated by simulation or real-world driving. To circumvent these limitations, we propose a new learning-based method to tackle the autonomous overtaking problem. We evaluate our approach in the popular car racing game Gran Turismo Sport, which is known for its detailed modeling of various cars and tracks. By leveraging curriculum learning, our approach leads to faster convergence as well as increased performance compared to vanilla reinforcement learning. As a result, the trained controller outperforms the built-in model-based game AI and achieves comparable overtaking performance with an experienced human driver.