# Physics-Informed Reinforcement Learning of Spatial Density Velocity Potentials for Map-Free Racing

arXiv: https://arxiv.org/abs/2604.09499 · HTML: https://mlsanity.com/paper/2604.09499/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods.
- Builds on 22 of its 35 references that are in the map.
- First posted on arXiv 10 Apr 2026.

## Placed in

- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 22 of 22)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusion](https://mlsanity.com/paper/2508.08241/) (2025)
- [OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interaction](https://mlsanity.com/paper/2509.26633/) (2025)
- [Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement Learning](https://mlsanity.com/paper/2008.07971/) (2020)
- [Autonomous Vehicles on the Edge: A Survey on Autonomous Vehicle Racing](https://mlsanity.com/paper/2202.07008/) (2022)
- [Optimization-Based Autonomous Racing of 1:43 Scale RC Cars](https://mlsanity.com/paper/1711.07300/) (2017)
- [Latent Imagination Facilitates Zero-Shot Transfer in Autonomous Racing](https://mlsanity.com/paper/2103.04909/) (2021)
- [VIRAL: Visual Sim-to-Real at Scale for Humanoid Loco-Manipulation](https://mlsanity.com/paper/2511.15200/) (2025)
- [Learning from Simulation, Racing in Reality](https://mlsanity.com/paper/2011.13332/) (2020)
- [TUM autonomous motorsport: An autonomous racing software for the Indy Autonomous Challenge](https://mlsanity.com/paper/2205.15979/) (2022)
- [Optimization-Based Hierarchical Motion Planning for Autonomous Racing](https://mlsanity.com/paper/2003.04882/) (2020)
- [High-speed Autonomous Racing using Trajectory-aided Deep Reinforcement Learning](https://mlsanity.com/paper/2306.07003/) (2023)
- [AutoDRIVE Simulator: A Simulator for Scaled Autonomous Vehicle Research and Education](https://mlsanity.com/paper/2103.10030/) (2021)
- [RaceMOP: Mapless Online Path Planning for Multi-Agent Autonomous Racing using Residual Policy Learning](https://mlsanity.com/paper/2403.07129/) (2024)
- [TinyLidarNet: 2D LiDAR-based End-to-End Deep Learning Model for F1TENTH Autonomous Racing](https://mlsanity.com/paper/2410.07447/) (2024)
- [Agile and Generalized Legged Locomotion via Attention-Based Neural Map Encoding](https://mlsanity.com/paper/2601.08485/) (2026)
- [XTENTH-CAR: A Proportionally Scaled Experimental Vehicle Platform for Connected Autonomy and All-Terrain Research](https://mlsanity.com/paper/2212.01691/) (2022)
- [AutoVRL: A High Fidelity Autonomous Ground Vehicle Simulator for Sim-to-Real Deep Reinforcement Learning](https://mlsanity.com/paper/2304.11496/) (2023)
- [A Nonlinear Model Predictive Control Strategy for Autonomous Racing of Scale Vehicles](https://mlsanity.com/paper/2302.04722/) (2023)
- [Advancing Autonomous Racing: A Comprehensive Survey of the RoboRacer (F1TENTH) Platform](https://mlsanity.com/paper/2506.15899/) (2025)

## Built on by (most cited first, 1 of 1)

- [Autonomous Vehicle Collision Avoidance With Racing Parameterized Deep Reinforcement Learning](https://mlsanity.com/paper/2604.16702/) (2026)

## Abstract (arXiv)

Autonomous racing without prebuilt maps is a grand challenge for embedded robotics that requires kinodynamic planning from instantaneous sensor data at the acceleration and tire friction limits. Out-Of-Distribution (OOD) generalization to various racetrack configurations utilizes Machine Learning (ML) to encode the mathematical relation between sensor data and vehicle actuation for end-to-end control, with implicit localization. These comprise Behavioral Cloning (BC) that is capped to human reaction times and Deep Reinforcement Learning (DRL) which requires large-scale collisions for comprehensive training that can be infeasible without simulation but is arduous to transfer to reality, thus exhibiting greater performance than BC in simulation, but actuation instability on hardware. This paper presents a DRL method that parameterizes nonlinear vehicle dynamics from the spectral distribution of depth measurements with a non-geometric, physics-informed reward, to infer vehicle time-optimal and overtaking racing controls with an Artificial Neural Network (ANN) that utilizes less than 1% of the computation of BC and model-based DRL. Slaloming from simulation to reality transfer and variance-induced conservatism are eliminated with the combination of a physics engine exploit-aware reward and the replacement of an explicit collision penalty with an implicit truncation of the value horizon. The policy outperforms human demonstrations by 12% in OOD tracks on proportionally scaled hardware, by maximizing the friction circle with tire dynamics that resemble an empirical Pacejka tire model. System identification illuminates a functional bifurcation where the first layer compresses spatial observations to extract digitized track features with higher resolution in corner apexes, and the second encodes nonlinear dynamics.