# iLLM-TSC: Integration reinforcement learning and large language model for traffic signal control policy improvement

arXiv: https://arxiv.org/abs/2407.06025 · HTML: https://mlsanity.com/paper/2407.06025/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 61 references that are in the map.
- First posted on arXiv 8 Jul 2024.

## Builds on (most cited first, 26 of 26)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023)
- [Eureka: Human-Level Reward Design via Coding Large Language Models](https://mlsanity.com/paper/2310.12931/) (2023)
- [DriveLM: Driving with Graph Visual Question Answering](https://mlsanity.com/paper/2312.14150/) (2023)
- [A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications](https://mlsanity.com/paper/2402.07927/) (2024)
- [GPT-Driver: Learning to Drive with GPT](https://mlsanity.com/paper/2310.01415/) (2023)
- [Emergent Analogical Reasoning in Large Language Models](https://mlsanity.com/paper/2212.09196/) (2022)
- [Drive Like a Human: Rethinking Autonomous Driving with Large Language Models](https://mlsanity.com/paper/2307.07162/) (2023)
- [Continual Learning for Large Language Models: A Survey](https://mlsanity.com/paper/2402.01364/) (2024)
- [Black-Box Prompt Optimization: Aligning Large Language Models without Model Training](https://mlsanity.com/paper/2311.04155/) (2023)
- [Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods](https://mlsanity.com/paper/2404.00282/) (2024)
- [Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey](https://mlsanity.com/paper/2404.01869/) (2024)
- [Learning to Model the World with Language](https://mlsanity.com/paper/2308.01399/) (2023)
- [AttendLight: Universal Attention-Based Reinforcement Learning Model for Traffic Signal Control](https://mlsanity.com/paper/2010.05772/) (2020)
- [Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraft](https://mlsanity.com/paper/2312.09238/) (2023)
- [TELeR: A General Taxonomy of LLM Prompts for Benchmarking Complex Tasks](https://mlsanity.com/paper/2305.11430/) (2023)
- [Open-TI: Open Traffic Intelligence with Augmented Language Model](https://mlsanity.com/paper/2401.00211/) (2023)
- [LLM-Assisted Light: Leveraging Large Language Model Capabilities for Human-Mimetic Traffic Signal Control in Complex Urban Environments](https://mlsanity.com/paper/2403.08337/) (2024)
- [Unleashing the Power of Pre-trained Language Models for Offline Reinforcement Learning](https://mlsanity.com/paper/2310.20587/) (2023)
- [UniTSA: A Universal Reinforcement Learning Framework for V2X Traffic Signal Control](https://mlsanity.com/paper/2312.05090/) (2023)
- [EMVLight: a Multi-agent Reinforcement Learning Framework for an Emergency Vehicle Decentralized Routing and Traffic Signal Control System](https://mlsanity.com/paper/2206.13441/) (2022)
- [Traffic Signal Cycle Control with Centralized Critic and Decentralized Actors under Varying Intervention Frequencies](https://mlsanity.com/paper/2406.08248/) (2024)
- [ADLight: A Universal Approach of Traffic Signal Control with Augmented Data Using Reinforcement Learning](https://mlsanity.com/paper/2210.13378/) (2022)

## Built on by (most cited first, 23 of 23)

- [Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmap](https://mlsanity.com/paper/2503.21411/) (2025)
- [Large Language Models (LLMs) as Traffic Control Systems at Urban Intersections: A New Paradigm](https://mlsanity.com/paper/2411.10869/) (2024)
- [Large Language Models and Their Applications in Roadway Safety and Mobility Enhancement: A Comprehensive Review](https://mlsanity.com/paper/2506.06301/) (2025)
- [Leveraging Multimodal-LLMs Assisted by Instance Segmentation for Intelligent Traffic Monitoring](https://mlsanity.com/paper/2502.11304/) (2025)
- [Large Language Model-Enhanced Reinforcement Learning for Generic Bus Holding Control Strategies](https://mlsanity.com/paper/2410.10212/) (2024)
- [Large Language Model Powered Intelligent Urban Agents: Concepts, Capabilities, and Applications](https://mlsanity.com/paper/2507.00914/) (2025)
- [STLLM-DF: A Spatial-Temporal Large Language Model with Diffusion for Enhanced Multi-Mode Traffic System Forecasting](https://mlsanity.com/paper/2409.05921/) (2024)
- [CoT-VLM4Tar: Chain-of-Thought Guided Vision-Language Models for Traffic Anomaly Resolution](https://mlsanity.com/paper/2503.01632/) (2025)
- [VLMLight: Safety-Critical Traffic Signal Control via Vision-Language Meta-Control and Dual-Branch Reasoning Architecture](https://mlsanity.com/paper/2505.19486/) (2025)
- [Urban Computing in the Era of Large Language Models](https://mlsanity.com/paper/2504.02009/) (2025)
- [A Dual Large Language Models Architecture with Herald Guided Prompts for Parallel Fine Grained Traffic Signal Control](https://mlsanity.com/paper/2511.00136/) (2025)
- [Virtual Traffic Police: Large Language Model-Augmented Traffic Signal Control for Unforeseen Incidents](https://mlsanity.com/paper/2601.15816/) (2026)
- [Visual Reasoning at Urban Intersections: FineTuning GPT-4o for Traffic Conflict Detection](https://mlsanity.com/paper/2502.20573/) (2025)
- [Accelerating Reinforcement Learning Algorithms Convergence using Pre-trained Large Language Models as Tutors With Advice Reusing](https://mlsanity.com/paper/2509.08329/) (2025)
- [Retrieval Augmented Generation-Enhanced Distributed LLM Agents for Generalizable Traffic Signal Control with Emergency Vehicles](https://mlsanity.com/paper/2510.26242/) (2025)
- [DGLight: DQN-Guided GRPO Fine-Tuning of Large Language Models for Traffic Signal Control](https://mlsanity.com/paper/2604.25259/) (2026)
- [LIDSA: Cognitive Arbitration for Signal-Free Autonomous Intersection Management](https://mlsanity.com/paper/2605.12321/) (2026)
- [Earth Science Foundation Models: From Perception to Reasoning and Discovery](https://mlsanity.com/paper/2605.12542/) (2026)
- [ReasonLight: A Multimodal Foundation Model-Enhanced Reinforcement Learning Framework for Zero-Shot Traffic Signal Control](https://mlsanity.com/paper/2605.29425/) (2026)
- [Large Language Models in Transportation Systems Management and Operations: From Text Reasoning to Multi-modal Decision Support](https://mlsanity.com/paper/2606.00991/) (2026)
- [A MARL Centered Reference Architecture for Large Language Model Augmentation in Smart Manufacturing](https://mlsanity.com/paper/2608.07148/) (2026)
- [Approved Too Late: Verdict Staleness in LLM-Guarded Self-Adaptive Systems](https://mlsanity.com/paper/2608.26306/) (2026)
- [VLALight: A Vision-Language-Action Model for Traffic Signal Control](https://mlsanity.com/paper/2609.36934/) (2026)

## Abstract (arXiv)

Urban congestion remains a critical challenge, with traffic signal control (TSC) emerging as a potent solution. TSC is often modeled as a Markov Decision Process problem and then solved using reinforcement learning (RL), which has proven effective. However, the existing RL-based TSC system often overlooks imperfect observations caused by degraded communication, such as packet loss, delays, and noise, as well as rare real-life events not included in the reward function, such as unconsidered emergency vehicles. To address these limitations, we introduce a novel integration framework that combines a large language model (LLM) with RL. This framework is designed to manage overlooked elements in the reward function and gaps in state information, thereby enhancing the policies of RL agents. In our approach, RL initially makes decisions based on observed data. Subsequently, LLMs evaluate these decisions to verify their reasonableness. If a decision is found to be unreasonable, it is adjusted accordingly. Additionally, this integration approach can be seamlessly integrated with existing RL-based TSC systems without necessitating modifications. Extensive testing confirms that our approach reduces the average waiting time by 17.5% in degraded communication conditions as compared to traditional RL methods, underscoring its potential to advance practical RL applications in intelligent transportation systems. The related code can be found at urlhttps://github.com/Traffic-Alpha/iLLM-TSC.