# Prompt to Transfer: Sim-to-Real Transfer for Traffic Signal Control with Prompt Learning

arXiv: https://arxiv.org/abs/2308.14284 · HTML: https://mlsanity.com/paper/2308.14284/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 37 references that are in the map.
- First posted on arXiv 28 Aug 2023.

## Builds on (most cited first, 18 of 18)

- [How Can We Know What Language Models Know?](https://mlsanity.com/paper/1911.12543/) (2019)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [Deep Domain Confusion: Maximizing for Domain Invariance](https://mlsanity.com/paper/1412.3474/) (2014)
- [Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey](https://mlsanity.com/paper/2009.13303/) (2020)
- [Robots that can adapt like animals](https://mlsanity.com/paper/1407.3501/) (2014)
- [Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks](https://mlsanity.com/paper/1812.07252/) (2018)
- [Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping](https://mlsanity.com/paper/1709.07857/) (2017)
- [CoLight: Learning Network-level Cooperation for Traffic Signal Control](https://mlsanity.com/paper/1905.05717/) (2019)
- [CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario](https://mlsanity.com/paper/1905.05217/) (2019)
- [Deep Reinforcement Learning for Intelligent Transportation Systems: A Survey](https://mlsanity.com/paper/2005.00935/) (2020)
- [Learning Phase Competition for Traffic Signal Control](https://mlsanity.com/paper/1905.04722/) (2019)
- [Multi-Task Domain Adaptation for Deep Learning of Instance Grasping from Simulation](https://mlsanity.com/paper/1710.06422/) (2017)
- [Honor of Kings Arena: an Environment for Generalization in Competitive Reinforcement Learning](https://mlsanity.com/paper/2209.08483/) (2022)
- [Reinforced Grounded Action Transformation for Sim-to-Real Transfer](https://mlsanity.com/paper/2008.01279/) (2020)
- [GeneraLight: Improving Environment Generalization of Traffic Signal Control via Meta Reinforcement Learning](https://mlsanity.com/paper/2009.08052/) (2020)
- [Stochastic Grounded Action Transformation for Robot Learning in Simulation](https://mlsanity.com/paper/2008.01281/) (2020)
- [Towards Real-World Deployment of Reinforcement Learning for Traffic Signal Control](https://mlsanity.com/paper/2103.16223/) (2021)
- [Deep Reinforcement Q-Learning for Intelligent Traffic Signal Control with Partial Detection](https://mlsanity.com/paper/2109.14337/) (2021)

## Built on by (most cited first, 30 of 36)

- [Uncertainty Quantification and Confidence Calibration in Large Language Models: A Survey](https://mlsanity.com/paper/2503.15850/) (2025)
- [TrafficGPT: Viewing, Processing and Interacting with Traffic Foundation Models](https://mlsanity.com/paper/2309.06719/) (2023)
- [A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Models](https://mlsanity.com/paper/2502.13187/) (2025)
- [ChatSUMO: Large Language Model for Automating Traffic Scenario Generation in Simulation of Urban MObility](https://mlsanity.com/paper/2409.09040/) (2024)
- [Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmap](https://mlsanity.com/paper/2503.21411/) (2025)
- [eTraM: Event-based Traffic Monitoring Dataset](https://mlsanity.com/paper/2403.19976/) (2024)
- [LLM Uncertainty Quantification through Directional Entailment Graph and Claim Level Response Augmentation](https://mlsanity.com/paper/2407.00994/) (2024)
- [Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directions](https://mlsanity.com/paper/2501.04437/) (2025)
- [CoMAL: Collaborative Multi-Agent Large Language Models for Mixed-Autonomy Traffic](https://mlsanity.com/paper/2410.14368/) (2024)
- [Towards Explainable Traffic Flow Prediction with Large Language Models](https://mlsanity.com/paper/2404.02937/) (2024)
- [Unraveling Spatio-Temporal Foundation Models via the Pipeline Lens: A Comprehensive Review](https://mlsanity.com/paper/2506.01364/) (2025)
- [Generative AI in Transportation Planning: A Survey](https://mlsanity.com/paper/2503.07158/) (2025)
- [CoSLight: Co-optimizing Collaborator Selection and Decision-making to Enhance Traffic Signal Control](https://mlsanity.com/paper/2405.17152/) (2024)
- [Re-boosting Self-Collaboration Parallel Prompt GAN for Unsupervised Image Restoration](https://mlsanity.com/paper/2408.09241/) (2024)
- [MalLight: Influence-Aware Coordinated Traffic Signal Control for Traffic Signal Malfunctions](https://mlsanity.com/paper/2408.09768/) (2024)
- [A Survey of Foundation Models for IoT: Taxonomy and Criteria-Based Analysis](https://mlsanity.com/paper/2506.12263/) (2025)
- [DeepShade: Enable Shade Simulation by Text-conditioned Image Generation](https://mlsanity.com/paper/2507.12103/) (2025)
- [Conformal Feedback Alignment: Quantifying Answer-Level Reliability for Robust LLM Alignment](https://mlsanity.com/paper/2601.17329/) (2026)
- [Probabilistic Offline Policy Ranking with Approximate Bayesian Computation](https://mlsanity.com/paper/2312.11551/) (2023)
- [A Simple Framework Towards Vision-based Traffic Signal Control with Microscopic Simulation](https://mlsanity.com/paper/2403.06884/) (2024)
- [Generalized Phase Pressure Control Enhanced Reinforcement Learning for Traffic Signal Control](https://mlsanity.com/paper/2503.20205/) (2025)
- [Urban Computing in the Era of Large Language Models](https://mlsanity.com/paper/2504.02009/) (2025)
- [cMALC-D: Contextual Multi-Agent LLM-Guided Curriculum Learning with Diversity-Based Context Blending](https://mlsanity.com/paper/2508.20818/) (2025)
- [A Dual Large Language Models Architecture with Herald Guided Prompts for Parallel Fine Grained Traffic Signal Control](https://mlsanity.com/paper/2511.00136/) (2025)
- [LLM-Guided Communication for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2605.18077/) (2026)
- [Agentic Episodic Control](https://mlsanity.com/paper/2506.01442/) (2025)
- [Virtual Traffic Police: Large Language Model-Augmented Traffic Signal Control for Unforeseen Incidents](https://mlsanity.com/paper/2601.15816/) (2026)
- [LATS: Large Language Model Assisted Teacher-Student Framework for Multi-Agent Reinforcement Learning in Traffic Signal Control](https://mlsanity.com/paper/2603.24361/) (2026)
- [The Sim-to-Real Gap of Foundation Model Agents: A Unified MDP Perspective](https://mlsanity.com/paper/2606.07017/) (2026)
- [Responsible AI in NLP: GUS-Net Span-Level Bias Detection Dataset and Benchmark for Generalizations, Unfairness, and Stereotypes](https://mlsanity.com/paper/2410.08388/) (2024)

## Abstract (arXiv)

Numerous solutions are proposed for the Traffic Signal Control (TSC) tasks aiming to provide efficient transportation and mitigate congestion waste. In recent, promising results have been attained by Reinforcement Learning (RL) methods through trial and error in simulators, bringing confidence in solving cities' congestion headaches. However, there still exist performance gaps when simulator-trained policies are deployed to the real world. This issue is mainly introduced by the system dynamic difference between the training simulator and the real-world environments. The Large Language Models (LLMs) are trained on mass knowledge and proved to be equipped with astonishing inference abilities. In this work, we leverage LLMs to understand and profile the system dynamics by a prompt-based grounded action transformation. Accepting the cloze prompt template, and then filling in the answer based on accessible context, the pre-trained LLM's inference ability is exploited and applied to understand how weather conditions, traffic states, and road types influence traffic dynamics, being aware of this, the policies' action is taken and grounded based on realistic dynamics, thus help the agent learn a more realistic policy. We conduct experiments using DQN to show the effectiveness of the proposed PromptGAT's ability in mitigating the performance gap from simulation to reality (sim-to-real).