# Self-Paced Curriculum Reinforcement Learning for Autonomous Superbike Racing in Simulation

arXiv: https://arxiv.org/abs/2606.09236 · HTML: https://mlsanity.com/paper/2606.09236/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods.
- Builds on 9 of its 12 references that are in the map.
- First posted on arXiv 8 Jun 2026.

## Placed in

- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 9 of 9)

- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Unity: A General Platform for Intelligent Agents](https://mlsanity.com/paper/1809.02627/) (2018)
- [Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement Learning](https://mlsanity.com/paper/2008.07971/) (2020)
- [Self-Paced Deep Reinforcement Learning](https://mlsanity.com/paper/2004.11812/) (2020)
- [Autonomous Overtaking in Gran Turismo Sport Using Curriculum Reinforcement Learning](https://mlsanity.com/paper/2103.14666/) (2021)
- [A Probabilistic Interpretation of Self-Paced Learning with Applications to Reinforcement Learning](https://mlsanity.com/paper/2102.13176/) (2021)
- [A Super-human Vision-based Reinforcement Learning Agent for Autonomous Racing in Gran Turismo](https://mlsanity.com/paper/2406.12563/) (2024)
- [Out-of-Distribution Generalization with a SPARC: Racing 100 Unseen Vehicles with a Single Policy](https://mlsanity.com/paper/2511.09737/) (2025)
- [A Champion-level Vision-based Reinforcement Learning Agent for Competitive Racing in Gran Turismo 7](https://mlsanity.com/paper/2504.09021/) (2025)

## Abstract (arXiv)

Autonomous Racing has seen remarkable progress through deep Reinforcement Learning (RL), primarily for four-wheeled vehicles. However, motorbikes introduce substantially greater complexity due to the need to manage balance and lean angle, in addition to more reactive steering and throttle control, and a smaller weight. In this work, we present a framework for training an autonomous agent to race a superbike in VRider SBK, a physics-accurate Unity-based motorbike simulator. Our approach integrates Soft Actor-Critic (SAC) with Self-Paced curriculum Deep reinforcement Learning (SPDL), which dynamically generates progressively more challenging tasks based on the agent's performance, without requiring manual curriculum design. The agent's state space comprises proprioceptive features extended with lean-angle history, along with global track features via course points. The reward signal is shaped to encourage progress along the track while penalizing instability-inducing behaviors specific to two-wheeled dynamics. Preliminary experimental results demonstrate that SPDL outperforms SAC alone in training efficiency, lap time, and driving stability across multiple tracks and motorbike models, establishing a first baseline for RL-based autonomous motorbike racing.