mlsanityData to 7 Oct 2026

CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario

arXiv 1905.05217 ↗First posted 13 May 2019cs.MA cs.LGcited by 114

Not on the map. Why?

Abstract (arXiv)

Traffic signal control is an emerging application scenario for reinforcement learning. Besides being as an important problem that affects people's daily life in commuting, traffic signal control poses its unique challenges for reinforcement learning in terms of adapting to dynamic traffic environment and coordinating thousands of agents including vehicles and pedestrians. A key factor in the success of modern reinforcement learning relies on a good simulator to generate a large number of data samples for learning. The most commonly used open-source traffic simulator SUMO is, however, not scalable to large road network and large traffic flow, which hinders the study of reinforcement learning on traffic scenarios. This motivates us to create a new traffic simulator CityFlow with fundamentally optimized data structures and efficient algorithms. CityFlow can support flexible definitions for road network and traffic flow based on synthetic and real-world data. It also provides user-friendly interface for reinforcement learning. Most importantly, CityFlow is more than twenty times faster than SUMO and is capable of supporting city-wide traffic simulation with an interactive render for monitoring. Besides traffic signal control, CityFlow could serve as the base for other transportation studies and can create new possibilities to test machine learning methods in the intelligent transportation domain.

Key facts. Cited by 114 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 17; 2022: 17; 2023: 23; 2024: 20; 2025: 26; 2026: 11.

114 papers.

Sort:

By area:

  1. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 281
  2. 2025AgentSociety: Large-Scale Simulation of LLM-Driven Generative Agents Advances Understanding of Human Behaviors and Societycited by 180
  3. 2024Embodied LLM Agents Learn to Cooperate in Organized Teamscited by 81
  4. 2022A Survey on Model-based Reinforcement Learningcited by 65
  5. 2023Open-TI: Open Traffic Intelligence with Augmented Language Modelcited by 38
  6. 2023Prompt to Transfer: Sim-to-Real Transfer for Traffic Signal Control with Prompt Learningcited by 36
  7. 2025A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Modelscited by 36
  8. 2021Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimizationcited by 33
  9. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 30
  10. 2021Expression might be enough: representing pressure and demand for reinforcement learning based traffic signal controlcited by 28
  11. 2023Choose Your Simulator Wisely: A Review on Open-source Simulators for Autonomous Drivingcited by 26
  12. 2024CityBench: Evaluating the Capabilities of Large Language Models for Urban Taskscited by 26
38 more
  1. 2021Efficient Pressure: Improving efficiency for signalized intersectionscited by 24
  2. 2022An Introduction to Multi-Agent Reinforcement Learning and Review of its Application to Autonomous Mobilitycited by 16
  3. 2022LibSignal: An Open Library for Traffic Signal Controlcited by 16
  4. 2025CoLLMLight: Cooperative Large Language Model Agents for Network-Wide Traffic Signal Controlcited by 15
  5. 2025Multi-Agent Reinforcement Learning for Resources Allocation Optimization: A Surveycited by 14
  6. 2023Reinforcement Learning Approaches for Traffic Signal Control under Missing Datacited by 13
  7. 2024FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learningcited by 13
  8. 2022Modeling Network-level Traffic Flow Transitions on Sparse Datacited by 12
  9. 2021DynSTGAT: Dynamic Spatial-Temporal Graph Attention Network for Traffic Signal Controlcited by 11
  10. 2022Multi-Agent Reinforcement Learning for Traffic Signal Control through Universal Communication Methodcited by 10
  11. 2023TransWorldNG: Traffic Simulation via Foundation Modelcited by 9
  12. 2023IMP-MARL: a Suite of Environments for Large-scale Infrastructure Management Planning via MARLcited by 9
  13. 2021Objective-aware Traffic Simulation via Inverse Reinforcement Learningcited by 8
  14. 2022Consensus Learning for Cooperative Multi-Agent Reinforcement Learningcited by 8
  15. 2022The Real Deal: A Review of Challenges and Opportunities in Moving Reinforcement Learning-Based Traffic Signal Control Systems Towards Realitycited by 8
  16. 2021Space Meets Time: Local Spacetime Neural Network For Traffic Flow Forecastingcited by 7
  17. 2021Leveraging Queue Length and Attention Mechanisms for Enhanced Traffic Signal Control Optimizationcited by 7
  18. 2023Off-the-Grid MARL: Datasets with Baselines for Offline Multi-Agent Reinforcement Learningcited by 7
  19. 2023DenseLight: Efficient Control for Large-scale Traffic Signals with Dense Feedbackcited by 7
  20. 2023FusionPlanner: A Multi-task Motion Planner for Mining Trucks via Multi-sensor Fusioncited by 7
  21. 2021MetaVIM: Meta Variationally Intrinsic Motivated Reinforcement Learning for Decentralized Traffic Signal Controlcited by 6
  22. 2021Learning to Simulate on Sparse Trajectory Datacited by 6
  23. 2022CBLab: Supporting the Training of Large-scale Traffic Control Policies with Scalable Traffic Simulationcited by 6
  24. 2023Large-Scale Traffic Signal Control Using Constrained Network Partition and Adaptive Deep Reinforcement Learningcited by 6
  25. 2023From Model-Based to Data-Driven Simulation: Challenges and Trends in Autonomous Drivingcited by 6
  26. 2024Heterogeneous Multi-Agent Reinforcement Learning for Zero-Shot Scalable Collaborationcited by 6
  27. 2021Independent Reinforcement Learning for Weakly Cooperative Multiagent Traffic Control Problemcited by 5
  28. 2022Offline Reinforcement Learning for Road Traffic Controlcited by 5
  29. 2024Towards Multi-agent Reinforcement Learning based Traffic Signal Control through Spatio-temporal Hypergraphscited by 5
  30. 2024MalLight: Influence-Aware Coordinated Traffic Signal Control for Traffic Signal Malfunctionscited by 5
  31. 2025Fox in the Henhouse: Supply-Chain Backdoor Attacks Against Reinforcement Learningcited by 5
  32. 2021Two-stage training algorithm for AI robot soccercited by 4
  33. 2021Learning to Share in Multi-Agent Reinforcement Learningcited by 4
  34. 2024CityFlowER: An Efficient and Realistic Traffic Simulator with Embedded Machine Learning Modelscited by 4
  35. 2024SynTraC: A Synthetic Dataset for Traffic Signal Control from Traffic Monitoring Camerascited by 4
  36. 2024Putting Data at the Centre of Offline Multi-Agent Reinforcement Learningcited by 4
  37. 2024DiffLight: A Partial Rewards Conditioned Diffusion Model for Traffic Signal Control with Missing Datacited by 4
  38. 2024Goal-Conditioned Data Augmentation for Offline Reinforcement Learningcited by 4

50 most cited of 114 All 114 (JSON)