# RoaD: Rollouts as Demonstrations for Closed-Loop Supervised Fine-Tuning of Autonomous Driving Policies

arXiv: https://arxiv.org/abs/2512.01993 · HTML: https://mlsanity.com/paper/2512.01993/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving.
- Builds on 32 of its 43 references that are in the map.
- First posted on arXiv 1 Dec 2025.

## Placed in

- End-to-end autonomous driving / [Imitation-learning end-to-end planners](https://mlsanity.com/driving/imitation-learning-end-to-end-planners/)

## Builds on (most cited first, 30 of 32)

- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles](https://mlsanity.com/paper/2106.11810/) (2021)
- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models](https://mlsanity.com/paper/2402.12289/) (2024)
- [ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst](https://mlsanity.com/paper/1812.03079/) (2018)
- [Exploring the Limitations of Behavior Cloning for Autonomous Driving](https://mlsanity.com/paper/1904.08980/) (2019)
- [AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning](https://mlsanity.com/paper/2506.13757/) (2025)
- [Deep Reinforcement Learning framework for Autonomous Driving](https://mlsanity.com/paper/1704.02532/) (2017)
- [DART: Noise Injection for Robust Imitation Learning](https://mlsanity.com/paper/1703.09327/) (2017)
- [Pseudo-Simulation for Autonomous Driving](https://mlsanity.com/paper/2506.04218/) (2025)
- [Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail](https://mlsanity.com/paper/2511.00088/) (2025)
- [Vision Language Models in Autonomous Driving: A Survey and Outlook](https://mlsanity.com/paper/2310.14414/) (2023)
- [Reinforcement and Imitation Learning via Interactive No-Regret Learning](https://mlsanity.com/paper/1406.5979/) (2014)
- [Interpretable End-to-end Urban Autonomous Driving with Latent Deep Reinforcement Learning](https://mlsanity.com/paper/2001.08726/) (2020)
- [Deeply AggreVaTeD: Differentiable Imitation Learning for Sequential Prediction](https://mlsanity.com/paper/1703.01030/) (2017)
- [3DGUT: Enabling Distorted Cameras and Secondary Rays in Gaussian Splatting](https://mlsanity.com/paper/2412.12507/) (2024)
- [The Waymo Open Sim Agents Challenge](https://mlsanity.com/paper/2305.12032/) (2023)
- [Query-Efficient Imitation Learning for End-to-End Autonomous Driving](https://mlsanity.com/paper/1605.06450/) (2016)
- [Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation](https://mlsanity.com/paper/1709.10489/) (2017)
- [A Survey on Vision-Language-Action Models for Autonomous Driving](https://mlsanity.com/paper/2506.24044/) (2025)
- [Safe, Efficient, and Comfortable Velocity Control based on Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/1902.00089/) (2019)
- [RAP: 3D Rasterization Augmented End-to-End Planning](https://mlsanity.com/paper/2510.04333/) (2025)
- [Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Models](https://mlsanity.com/paper/2412.05334/) (2024)
- [Reinforcement Learning for Autonomous Driving with Latent State Inference and Spatial-Temporal Relationships](https://mlsanity.com/paper/2011.04251/) (2020)
- [DiffStack: A Differentiable and Modular Control Stack for Autonomous Vehicles](https://mlsanity.com/paper/2212.06437/) (2022)
- [Driving in Dense Traffic with Model-Free Reinforcement Learning](https://mlsanity.com/paper/1909.06710/) (2019)
- [SMART: Scalable Multi-agent Real-time Motion Generation via Next-token Prediction](https://mlsanity.com/paper/2405.15677/) (2024)
- [Advancing Multi-agent Traffic Simulation via R1-Style Reinforcement Fine-Tuning](https://mlsanity.com/paper/2509.23993/) (2025)
- [UniMM: A Unified Mixture Model Framework for Multi-Agent Simulation](https://mlsanity.com/paper/2501.17015/) (2025)

## Built on by (most cited first, 6 of 6)

- [BridgeSim: Unveiling the OL-CL Gap in End-to-End Autonomous Driving](https://mlsanity.com/paper/2604.10856/) (2026)
- [Scaling Self-Play for End-to-End Driving](https://mlsanity.com/paper/2606.19641/) (2026)
- [Learning from Mistakes: Rollout-Retrieval Lifelong Policy Learning for Autonomous Driving](https://mlsanity.com/paper/2606.30537/) (2026)
- [OPTED: On-Policy Fine-Tuning for End-to-End Driving using a Render-Free Teacher](https://mlsanity.com/paper/2609.20756/) (2026)
- [Closed Loop Dynamic Driving Data Mixture for Real-Synthetic Co-Training](https://mlsanity.com/paper/2605.21372/) (2026)
- [Flow-ERD: Agent-type Aware Flow Matching with Entropy-Regularized Distillation for Diverse Traffic Simulation](https://mlsanity.com/paper/2607.06957/) (2026)

## Abstract (arXiv)

Autonomous driving policies are typically trained via open-loop behavior cloning of human demonstrations. However, such policies suffer from covariate shift when deployed in closed loop, leading to compounding errors. We introduce Rollouts as Demonstrations (RoaD), a simple and efficient method to mitigate covariate shift by leveraging the policy's own closed-loop rollouts as additional training data. During rollout generation, RoaD incorporates expert guidance to bias trajectories toward high-quality behavior, producing informative yet realistic demonstrations for fine-tuning. This approach enables robust closed-loop adaptation with orders of magnitude less data than reinforcement learning, and avoids restrictive assumptions of prior closed-loop supervised fine-tuning (CL-SFT) methods, allowing broader applications domains including end-to-end driving. We demonstrate the effectiveness of RoaD on WOSAC, a large-scale traffic simulation benchmark, where it performs similar or better than the prior CL-SFT method; and in AlpaSim, a high-fidelity neural reconstruction-based simulator for end-to-end driving, where it improves driving score by 41% and reduces collisions by 54%.