# Towards Safety Assured End-to-End Vision-Based Control for Autonomous Racing

arXiv: https://arxiv.org/abs/2303.02267 · HTML: https://mlsanity.com/paper/2303.02267/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Mar 2023.

## Built on by (most cited first, 6 of 6)

- [Enhance Planning with Physics-informed Safety Controller for End-to-end Autonomous Driving](https://mlsanity.com/paper/2405.00316/) (2024)
- [WROOM: An Autonomous Driving Approach for Off-Road Navigation](https://mlsanity.com/paper/2404.08855/) (2024)
- [Towards Optimal Head-to-head Autonomous Racing with Curriculum Reinforcement Learning](https://mlsanity.com/paper/2308.13491/) (2023)
- [Disturbance Observer-based Control Barrier Functions with Residual Model Learning for Safe Reinforcement Learning](https://mlsanity.com/paper/2410.06570/) (2024)
- [End-to-End Imitation Learning for Optimal Asteroid Proximity Operations](https://mlsanity.com/paper/2502.01034/) (2025)
- [Expert Knowledge-driven Reinforcement Learning for Autonomous Racing via Trajectory Guidance and Dynamics Constraints](https://mlsanity.com/paper/2603.05842/) (2026)

## Abstract (arXiv)

Autonomous car racing is a challenging task, as it requires precise applications of control while the vehicle is operating at cornering speeds. Traditional autonomous pipelines require accurate pre-mapping, localization, and planning which make the task computationally expensive and environment-dependent. Recent works propose use of imitation and reinforcement learning to train end-to-end deep neural networks and have shown promising results for high-speed racing. However, the end-to-end models may be dangerous to be deployed on real systems, as the neural networks are treated as black-box models devoid of any provable safety guarantees. In this work we propose a decoupled approach where an optimal end-to-end controller and a state prediction end-to-end model are learned together, and the predicted state of the vehicle is used to formulate a control barrier function for safeguarding the vehicle to stay within lane boundaries. We validate our algorithm both on a high-fidelity Carla driving simulator and a 1/10-scale RC car on a real track. The evaluation results suggest that using an explicit safety controller helps to learn the task safely with fewer iterations and makes it possible to safely navigate the vehicle on the track along the more challenging racing line.