# Expert Knowledge-driven Reinforcement Learning for Autonomous Racing via Trajectory Guidance and Dynamics Constraints

arXiv: https://arxiv.org/abs/2603.05842 · HTML: https://mlsanity.com/paper/2603.05842/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods.
- Builds on 16 of its 39 references that are in the map.
- First posted on arXiv 6 Mar 2026.

## Placed in

- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 16 of 16)

- [Control Barrier Functions: Theory and Applications](https://mlsanity.com/paper/1903.11199/) (2019)
- [Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes](https://mlsanity.com/paper/2408.03539/) (2024)
- [Exploration in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2205.00824/) (2022)
- [Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement Learning](https://mlsanity.com/paper/2008.07971/) (2020)
- [Autonomous Vehicles on the Edge: A Survey on Autonomous Vehicle Racing](https://mlsanity.com/paper/2202.07008/) (2022)
- [Autonomous Overtaking in Gran Turismo Sport Using Curriculum Reinforcement Learning](https://mlsanity.com/paper/2103.14666/) (2021)
- [TUM autonomous motorsport: An autonomous racing software for the Indy Autonomous Challenge](https://mlsanity.com/paper/2205.15979/) (2022)
- [Indy Autonomous Challenge -- Autonomous Race Cars at the Handling Limits](https://mlsanity.com/paper/2202.03807/) (2022)
- [High-speed Autonomous Racing using Trajectory-aided Deep Reinforcement Learning](https://mlsanity.com/paper/2306.07003/) (2023)
- [Hybrid Action Based Reinforcement Learning for Multi-Objective Compatible Autonomous Driving](https://mlsanity.com/paper/2501.08096/) (2025)
- [Towards Safety Assured End-to-End Vision-Based Control for Autonomous Racing](https://mlsanity.com/paper/2303.02267/) (2023)
- [A Sequential Two-Step Algorithm for Fast Generation of Vehicle Racing Trajectories](https://mlsanity.com/paper/1902.00606/) (2019)
- [Learning Model Predictive Control with Error Dynamics Regression for Autonomous Racing](https://mlsanity.com/paper/2309.10716/) (2023)
- [BeTAIL: Behavior Transformer Adversarial Imitation Learning from Human Racing Gameplay](https://mlsanity.com/paper/2402.14194/) (2024)
- [α-RACER: Real-Time Algorithm for Game-Theoretic Motion Planning and Control in Autonomous Racing using Near-Potential Function](https://mlsanity.com/paper/2412.08855/) (2024)
- [Online Velocity Profile Generation and Tracking for Sampling-Based Local Planning Algorithms in Autonomous Racing Environments](https://mlsanity.com/paper/2505.05157/) (2025)

## Abstract (arXiv)

Reinforcement learning has shown significant potential for autonomous racing, but it still faces challenges such as training instability, inefficient exploration, and unsafe action outputs in high-dynamic racing scenarios. This paper proposes a Trajectory guidance and Dynamics constraints Reinforcement Learning (TraD-RL) framework for autonomous racing. The proposed method incorporates expert prior knowledge into policy learning through Minimum Curvature Racing Line (MCRL) guidance, explicit vehicle dynamics constraints, and two-stage curriculum learning. MCRL provides global path and velocity references through observation augmentation and reward shaping, thereby improving exploration efficiency and racing performance. Yaw rate and sideslip angle constraints are introduced to characterize the vehicle dynamic safe operating envelope, and the corresponding stability costs are incorporated into policy optimization through Lagrangian relaxation. Moreover, the two-stage curriculum learning strategy enables a progressive transition from stable trajectory following to high-speed performance exploration. Experiments on two racetracks demonstrate that TraD-RL improves racing performance while maintaining a favorable balance between speed and dynamic stability. Further analyses of ablation, sensitivity, and robustness validate the effectiveness and stability of the proposed framework.