# Simulation to Scaled City: Zero-Shot Policy Transfer for Traffic Control via Autonomous Vehicles

arXiv: https://arxiv.org/abs/1812.06120 · HTML: https://mlsanity.com/paper/1812.06120/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Dec 2018.

## Built on by (most cited first, 19 of 19)

- [AD4RL: Autonomous Driving Benchmarks for Offline Reinforcement Learning with Value-based Dataset](https://mlsanity.com/paper/2404.02429/) (2024)
- [Unified Automatic Control of Vehicular Systems with Reinforcement Learning](https://mlsanity.com/paper/2208.00268/) (2022)
- [Learning to Control and Coordinate Mixed Traffic Through Robot Vehicles at Complex and Unsignalized Intersections](https://mlsanity.com/paper/2301.05294/) (2023)
- [Scalable Multiagent Driving Policies For Reducing Traffic Congestion](https://mlsanity.com/paper/2103.00058/) (2021)
- [Adversarial Deep Reinforcement Learning for Improving the Robustness of Multi-agent Autonomous Driving Policies](https://mlsanity.com/paper/2112.11937/) (2021)
- [Temporal Transfer Learning for Traffic Optimization with Coarse-grained Advisory Autonomy](https://mlsanity.com/paper/2312.09436/) (2023)
- [Autonomous Driving Small-Scale Cars: A Survey of Recent Development](https://mlsanity.com/paper/2404.06229/) (2024)
- [Ablation Study of How Run Time Assurance Impacts the Training and Performance of Reinforcement Learning Agents](https://mlsanity.com/paper/2207.04117/) (2022)
- [Evaluating the Robustness of Deep Reinforcement Learning for Autonomous Policies in a Multi-agent Urban Driving Environment](https://mlsanity.com/paper/2112.11947/) (2021)
- [Demonstrating Reinforcement Learning and Run Time Assurance for Spacecraft Inspection Using Unmanned Aerial Vehicles](https://mlsanity.com/paper/2405.06770/) (2024)
- [Mitigating Metropolitan Carbon Emissions with Dynamic Eco-driving at Scale](https://mlsanity.com/paper/2408.05609/) (2024)
- [Reinforcement Learning Based Oscillation Dampening: Scaling up Single-Agent RL algorithms to a 100 AV highway field operational test](https://mlsanity.com/paper/2402.17050/) (2024)
- [Learning to Change: Choreographing Mixed Traffic Through Lateral Control and Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2403.14879/) (2024)
- [ReMAV: Reward Modeling of Autonomous Vehicles for Finding Likely Failure Events](https://mlsanity.com/paper/2308.14550/) (2023)
- [Analyzing Behaviors of Mixed Traffic via Reinforcement Learning at Unsignalized Intersections](https://mlsanity.com/paper/2312.05325/) (2023)
- [Traffic Smoothing Controllers for Autonomous Vehicles Using Deep Reinforcement Learning and Real-World Trajectory Data](https://mlsanity.com/paper/2401.09666/) (2024)
- [Space Processor Computation Time Analysis for Reinforcement Learning and Run Time Assurance Control Policies](https://mlsanity.com/paper/2405.06771/) (2024)
- [Investigating the Impact of Choice on Deep Reinforcement Learning for Space Controls](https://mlsanity.com/paper/2405.12355/) (2024)
- [Optimizing Efficiency of Mixed Traffic through Reinforcement Learning: A Topology-Independent Approach and Benchmark](https://mlsanity.com/paper/2501.16728/) (2025)

## Abstract (arXiv)

Using deep reinforcement learning, we train control policies for autonomous vehicles leading a platoon of vehicles onto a roundabout. Using Flow, a library for deep reinforcement learning in micro-simulators, we train two policies, one policy with noise injected into the state and action space and one without any injected noise. In simulation, the autonomous vehicle learns an emergent metering behavior for both policies in which it slows to allow for smoother merging. We then directly transfer this policy without any tuning to the University of Delaware Scaled Smart City (UDSSC), a 1:25 scale testbed for connected and automated vehicles. We characterize the performance of both policies on the scaled city. We show that the noise-free policy winds up crashing and only occasionally metering. However, the noise-injected policy consistently performs the metering behavior and remains collision-free, suggesting that the noise helps with the zero-shot policy transfer. Additionally, the transferred, noise-injected policy leads to a 5% reduction of average travel time and a reduction of 22% in maximum travel time in the UDSSC. Videos of the controllers can be found at https://sites.google.com/view/iccps-policy-transfer.