# REARL: A Closed-loop Autonomous Driving Simulation Enhancement Framework with Real Traffic Data and Large Language Models

arXiv: https://arxiv.org/abs/2609.19903 · HTML: https://mlsanity.com/paper/2609.19903/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Large language models for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 28 of its 53 references that are in the map.
- First posted on arXiv 17 Sep 2026.

## Placed in

- End-to-end autonomous driving / [Large language models for driving](https://mlsanity.com/driving/large-language-models-for-driving/)

## Builds on (most cited first, 28 of 28)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [Explaining Explanations: An Overview of Interpretability of Machine Learning](https://mlsanity.com/paper/1806.00069/) (2018)
- [Exploring the Limitations of Behavior Cloning for Autonomous Driving](https://mlsanity.com/paper/1904.08980/) (2019)
- [A Survey on Neural Network Interpretability](https://mlsanity.com/paper/2012.14261/) (2020)
- [Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory](https://mlsanity.com/paper/2305.17144/) (2023)
- [The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems](https://mlsanity.com/paper/1810.05642/) (2018)
- [DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models](https://mlsanity.com/paper/2309.16292/) (2023)
- [Language Models of Code are Few-Shot Commonsense Learners](https://mlsanity.com/paper/2210.07128/) (2022)
- [LLM4Drive: A Survey of Large Language Models for Autonomous Driving](https://mlsanity.com/paper/2311.01043/) (2023)
- [A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective](https://mlsanity.com/paper/2202.02215/) (2022)
- [Milestones in Autonomous Driving and Intelligent Vehicles: Survey of Surveys](https://mlsanity.com/paper/2303.17220/) (2023)
- [Human-Like Autonomous Car-Following Model with Deep Reinforcement Learning](https://mlsanity.com/paper/1901.00569/) (2019)
- [A Physics-Informed Deep Learning Paradigm for Car-Following Models](https://mlsanity.com/paper/2012.13376/) (2020)
- [Towards Corner Case Detection for Autonomous Driving](https://mlsanity.com/paper/1902.09184/) (2019)
- [Deflating Dataset Bias Using Synthetic Data Augmentation](https://mlsanity.com/paper/2004.13866/) (2020)
- [An Application-Driven Conceptualization of Corner Cases for Perception in Highly Automated Driving](https://mlsanity.com/paper/2103.03678/) (2021)
- [Multi-Agent Autonomous Driving Systems with Large Language Models: A Survey of Recent Advances](https://mlsanity.com/paper/2502.16804/) (2025)
- [Enhancing Autonomous Vehicle Training with Language Model Integration and Critical Scenario Generation](https://mlsanity.com/paper/2404.08570/) (2024)
- [KoMA: Knowledge-driven Multi-agent Framework for Autonomous Driving with Large Language Models](https://mlsanity.com/paper/2407.14239/) (2024)
- [Large Language Models for Human-like Autonomous Driving: A Survey](https://mlsanity.com/paper/2407.19280/) (2024)
- [Extracting Traffic Primitives Directly from Naturalistically Logged Data for Self-Driving Applications](https://mlsanity.com/paper/1709.03553/) (2017)
- [Exploring the Potential of Large Language Models for Massive MIMO CSI Feedback](https://mlsanity.com/paper/2501.10630/) (2025)
- [Bringing Diversity to Autonomous Vehicles: An Interpretable Multi-vehicle Decision-making and Planning Framework](https://mlsanity.com/paper/2302.06803/) (2023)
- [LinguaSim: Interactive Multi-Vehicle Testing Scenario Generation via Natural Language Instruction Based on Large Language Models](https://mlsanity.com/paper/2510.08046/) (2025)
- [A Learning-based Stochastic Driving Model for Autonomous Vehicle Testing](https://mlsanity.com/paper/2102.02602/) (2021)
- [A Multi-Agent LLM Framework for Design Space Exploration in Autonomous Driving Systems](https://mlsanity.com/paper/2512.08476/) (2025)

## Abstract (arXiv)

Accurate simulation is crucial for autonomous driving development, yet capturing real-world traffic complexity remains challenging. Existing simulators that rely on predefined rules or static data playback struggle with dynamic traffic. CRITICAL uses real traffic data and a large language model (LLM) to adjust the initial simulation configuration, but the simulated distribution still diverges from real traffic as the rollout evolves. We propose REARL, a closed-loop simulation enhancement framework that integrates real traffic data with LLMs. Real traffic data are clustered, and each cluster center is used as a representative scenario that provides typical real-world traffic patterns for the LLM. A timed sliding-window detector then monitors discrepancies in vehicle speed distribution and mean spacing between pairs of vehicles. If a metric exceeds a threshold, the LLM adjusts vehicle decision-making; otherwise the existing controller is kept. The LLM also selects a matching real vehicle from a traffic snapshot and modulates the simulated vehicle with reference to that real action. In a controlled HighD highway setting, compared with the CRITICAL baseline and a PPO-based learning baseline, REARL reduces the Hellinger distance for speed distributions to 0.3067 and the MAPE for mean spacing to 0.8371, while achieving a time headway (THW) of 22.8575 and a lane change rate of 0.0708.