V-Max: A Reinforcement Learning Framework for Autonomous Driving
On the map: End-to-end autonomous driving → Reinforcement learning for driving
Abstract (arXiv)
Learning-based decision-making has the potential to enable generalizable Autonomous Driving (AD) policies, reducing the engineering overhead of rule-based approaches. Imitation Learning (IL) remains the dominant paradigm, benefiting from large-scale human demonstration datasets, but it suffers from inherent limitations such as distribution shift and imitation gaps. Reinforcement Learning (RL) presents a promising alternative, yet its adoption in AD remains limited due to the lack of standardized and efficient research frameworks. To this end, we introduce V-Max, an open research framework providing all the necessary tools to make RL practical for AD. V-Max is built on Waymax, a hardware-accelerated AD simulator designed for large-scale experimentation. We extend it using ScenarioNet's approach, enabling the fast simulation of diverse AD datasets.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 36 of its 48 references that are in the map. First posted on arXiv 11 Mar 2025.
Builds on
36 of 48 references.
Closest ancestors · 1
Same direction, newest first
Other references · 35
- 2017Attention Is All You Needcited by 59,932
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
- 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
- 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
- 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 701
- 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
- 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
- 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
- 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 532
- 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 485
23 more
- 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
- 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
- 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 375
- 2018Learning to Drive in a Daycited by 266
- 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 238
- 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Researchcited by 158
- 2019End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordancescited by 147
- 2023MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Queryingcited by 139
- 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarioscited by 124
- 2021Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradientscited by 122
- 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous drivingcited by 103
- 2023The Waymo Open Sim Agents Challengecited by 86
- 2025Robust Autonomy Emerges from Self-Playcited by 64
- 2023ScenarioNet: Open-Source Platform for Large-Scale Traffic Scenario Simulation and Modelingcited by 60
- 2022Nocturne: a scalable driving benchmark for bringing multi-agent learning one step closer to the real worldcited by 56
- 2021On Exposing the Challenging Long Tail in Future Prediction of Traffic Actorscited by 48
- 2023Scenario Diffusion: Controllable Driving Scenario Generation With Diffusioncited by 46
- 2022Hierarchical Model-Based Imitation Learning for Planning in Autonomous Drivingcited by 43
- 2024GPUDrive: Data-driven, multi-agent driving simulation at 1 million FPScited by 43
- 2018End-to-End Race Driving with Deep Reinforcement Learningcited by 38
- 2022Embedding Synthetic Off-Policy Experience for Autonomous Driving via Zero-Shot Curriculacited by 17
- 2024ReGentS: Real-World Safety-Critical Driving Scenario Generation Made Stablecited by 8
- 2024IGDrivSim: A Benchmark for the Imitation Gap in Autonomous Drivingcited by 3
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Beyond Self-Play and Scale: A Behavior Benchmark for Generalization in Autonomous Drivingcited by 5
- 2026HetroD: A High-Fidelity Drone Dataset and Benchmark for Autonomous Driving in Heterogeneous Trafficcited by 1
- 2026Reward-Conditioned Attention: How Reward Design Shapes What Autonomous Driving Agents Seecited by 0
- 2026Reward-Conditioned Attention: How Reward Design Shapes What Autonomous Driving Agents See23 Jun 2026
- 2026Beyond Self-Play and Scale: A Behavior Benchmark for Generalization in Autonomous Driving11 May 2026
- 2026HetroD: A High-Fidelity Drone Dataset and Benchmark for Autonomous Driving in Heterogeneous Traffic3 Feb 2026
Similar, not cited
Close in wording, no citation either way
- 2022Driver Dojo: A Benchmark for Generalizable Reinforcement Learning for Autonomous Driving
- 2025A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspective
- 2021Improved Deep Reinforcement Learning with Expert Demonstrations for Urban Autonomous Driving
- 2025A Comprehensive Review of Reinforcement Learning for Autonomous Driving in the CARLA Simulator
- 2024Reinforcement Learning for Online Testing of Autonomous Driving Systems: a Replication and Extension Study
- 2021A Versatile and Efficient Reinforcement Learning Framework for Autonomous Driving
- 2026Devil is in Narrow Policy: Unleashing Exploration in Driving VLA Models
- 2021A Survey of Deep RL and IL for Autonomous Driving Policy Learning