# Traffic Co-Simulation Framework Empowered by Infrastructure Camera Sensing and Reinforcement Learning

arXiv: https://arxiv.org/abs/2412.03925 · HTML: https://mlsanity.com/paper/2412.03925/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Multi-agent reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 19 of its 59 references that are in the map.
- First posted on arXiv 5 Dec 2024.

## Placed in

- Deep reinforcement learning methods / [Multi-agent reinforcement learning](https://mlsanity.com/deeprl/multi-agent-reinforcement-learning/)

## Builds on (most cited first, 19 of 19)

- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information](https://mlsanity.com/paper/2402.13616/) (2024)
- [Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control](https://mlsanity.com/paper/1903.04527/) (2019)
- [Safe, Efficient, and Comfortable Velocity Control based on Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/1902.00089/) (2019)
- [Simulation-based reinforcement learning for real-world autonomous driving](https://mlsanity.com/paper/1911.12905/) (2019)
- [ChatSUMO: Large Language Model for Automating Traffic Scenario Generation in Simulation of Urban MObility](https://mlsanity.com/paper/2409.09040/) (2024)
- [Traffic Light Control Using Deep Policy-Gradient and Value-Function Based Reinforcement Learning](https://mlsanity.com/paper/1704.08883/) (2017)
- [Large-Scale Traffic Signal Control Using a Novel Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1908.03761/) (2019)
- [Learning an Interpretable Traffic Signal Control Policy](https://mlsanity.com/paper/1912.11023/) (2019)
- [Cooperative Perception with Deep Reinforcement Learning for Connected Vehicles](https://mlsanity.com/paper/2004.10927/) (2020)
- [An Overview of Agent-based Traffic Simulators](https://mlsanity.com/paper/2102.07505/) (2021)
- [Deep Learning based Computer Vision Methods for Complex Traffic Environments Perception: A Review](https://mlsanity.com/paper/2211.05120/) (2022)
- [Smoothing Traffic Flow via Control of Autonomous Vehicles](https://mlsanity.com/paper/1812.09544/) (2018)
- [A Reinforcement Learning Approach for Intelligent Traffic Signal Control at Urban Intersections](https://mlsanity.com/paper/1905.07698/) (2019)
- [A Simple Framework Towards Vision-based Traffic Signal Control with Microscopic Simulation](https://mlsanity.com/paper/2403.06884/) (2024)
- [Field Deployment of Multi-Agent Reinforcement Learning Based Variable Speed Limit Controllers](https://mlsanity.com/paper/2407.08021/) (2024)
- [JADE, TraSMAPI and SUMO: A tool-chain for simulating traffic light control](https://mlsanity.com/paper/1601.08154/) (2016)
- [Customized Co-Simulation Environment for Autonomous Driving Algorithm Development and Evaluation](https://mlsanity.com/paper/2306.00223/) (2023)

## Built on by (most cited first, 1 of 1)

- [A Comprehensive Survey on Multi-Agent Cooperative Decision-Making: Scenarios, Approaches, Challenges and Perspectives](https://mlsanity.com/paper/2503.13415/) (2025)

## Abstract (arXiv)

Traffic simulations are commonly used to optimize urban traffic flow, with reinforcement learning (RL) showing promising potential for automated traffic signal control, particularly in intelligent transportation systems involving connected automated vehicles. Multi-agent reinforcement learning (MARL) is particularly effective for learning control strategies for traffic lights in a network using iterative simulations. However, existing methods often assume perfect vehicle detection, which overlooks real-world limitations related to infrastructure availability and sensor reliability. This study proposes a co-simulation framework integrating CARLA and SUMO, which combines high-fidelity 3D modeling with large-scale traffic flow simulation. Cameras mounted on traffic light poles within the CARLA environment use a YOLO-based computer vision system to detect and count vehicles, providing real-time traffic data as input for adaptive signal control in SUMO. MARL agents trained with four different reward structures leverage this visual feedback to optimize signal timings and improve network-wide traffic flow. Experiments in a multi-intersection test-bed demonstrate the effectiveness of the proposed MARL approach in enhancing traffic conditions using real-time camera based detection. The framework also evaluates the robustness of MARL under faulty or sparse sensing and compares the performance of YOLOv5 and YOLOv8 for vehicle detection. Results show that while better accuracy improves performance, MARL agents can still achieve significant improvements with imperfect detection, demonstrating scalability and adaptability for real-world scenarios.