# X-Light: Cross-City Traffic Signal Control Using Transformer on Transformer as Meta Multi-Agent Reinforcement Learner

arXiv: https://arxiv.org/abs/2404.12090 · HTML: https://mlsanity.com/paper/2404.12090/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 50 references that are in the map.
- First posted on arXiv 18 Apr 2024.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Decision Transformer: Reinforcement Learning via Sequence Modeling](https://mlsanity.com/paper/2106.01345/) (2021)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Offline Reinforcement Learning as One Big Sequence Modeling Problem](https://mlsanity.com/paper/2106.02039/) (2021)
- [Transformer in Transformer](https://mlsanity.com/paper/2103.00112/) (2021)
- [RL2: Fast Reinforcement Learning via Slow Reinforcement Learning](https://mlsanity.com/paper/1611.02779/) (2016)
- [Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control](https://mlsanity.com/paper/1903.04527/) (2019)
- [Prompting Decision Transformer for Few-Shot Policy Generalization](https://mlsanity.com/paper/2206.13499/) (2022)
- [CoLight: Learning Network-level Cooperation for Traffic Signal Control](https://mlsanity.com/paper/1905.05717/) (2019)
- [Learning Phase Competition for Traffic Signal Control](https://mlsanity.com/paper/1905.04722/) (2019)
- [Meta reinforcement learning as task inference](https://mlsanity.com/paper/1905.06424/) (2019)
- [AttendLight: Universal Attention-Based Reinforcement Learning Model for Traffic Signal Control](https://mlsanity.com/paper/2010.05772/) (2020)
- [Deep Reinforcement Learning for Traffic Light Control in Vehicular Networks](https://mlsanity.com/paper/1803.11115/) (2018)
- [Transformers are Meta-Reinforcement Learners](https://mlsanity.com/paper/2206.06614/) (2022)
- [Cascade Transformers for End-to-End Person Search](https://mlsanity.com/paper/2203.09642/) (2022)
- [Using Task Descriptions in Lifelong Machine Learning for Improved Performance and Zero-Shot Transfer](https://mlsanity.com/paper/1710.03850/) (2017)
- [GeneraLight: Improving Environment Generalization of Traffic Signal Control via Meta Reinforcement Learning](https://mlsanity.com/paper/2009.08052/) (2020)
- [DualFormer: Local-Global Stratified Transformer for Efficient Video Recognition](https://mlsanity.com/paper/2112.04674/) (2021)
- [DuaLight: Enhancing Traffic Signal Control by Leveraging Scenario-Specific and Scenario-Shared Knowledge](https://mlsanity.com/paper/2312.14532/) (2023)
- [MetaVIM: Meta Variationally Intrinsic Motivated Reinforcement Learning for Decentralized Traffic Signal Control](https://mlsanity.com/paper/2101.00746/) (2021)
- [PDiT: Interleaving Perception and Decision-making Transformers for Deep Reinforcement Learning](https://mlsanity.com/paper/2312.15863/) (2023)

## Built on by (most cited first, 10 of 10)

- [MalLight: Influence-Aware Coordinated Traffic Signal Control for Traffic Signal Malfunctions](https://mlsanity.com/paper/2408.09768/) (2024)
- [Unicorn: A Universal and Collaborative Reinforcement Learning Approach Towards Generalizable Network-Wide Traffic Signal Control](https://mlsanity.com/paper/2503.11488/) (2025)
- [GuideLight: "Industrial Solution" Guidance for More Practical Traffic Signal Control Agents](https://mlsanity.com/paper/2407.10811/) (2024)
- [LATS: Large Language Model Assisted Teacher-Student Framework for Multi-Agent Reinforcement Learning in Traffic Signal Control](https://mlsanity.com/paper/2603.24361/) (2026)
- [Transformer-based Drum-level Prediction in a Boiler Plant with Delayed Relations among Multivariates](https://mlsanity.com/paper/2407.11180/) (2024)
- [Exploring Genre and Success Classification through Song Lyrics using DistilBERT: A Fun NLP Venture](https://mlsanity.com/paper/2407.21068/) (2024)
- [HALO: Hierarchical Reinforcement Learning for Large-Scale Adaptive Traffic Signal Control](https://mlsanity.com/paper/2506.14391/) (2025)
- [Joint-Local Grounded Action Transformation for Sim-to-Real Transfer in Multi-Agent Traffic Control](https://mlsanity.com/paper/2507.15174/) (2025)
- [Spatiotemporal Decision Transformer for Traffic Coordination](https://mlsanity.com/paper/2602.02903/) (2026)
- [Emergency Preemption Without Online Exploration: A Decision Transformer Approach](https://mlsanity.com/paper/2603.22315/) (2026)

## Abstract (arXiv)

The effectiveness of traffic light control has been significantly improved by current reinforcement learning-based approaches via better cooperation among multiple traffic lights. However, a persisting issue remains: how to obtain a multi-agent traffic signal control algorithm with remarkable transferability across diverse cities? In this paper, we propose a Transformer on Transformer (TonT) model for cross-city meta multi-agent traffic signal control, named as X-Light: We input the full Markov Decision Process trajectories, and the Lower Transformer aggregates the states, actions, rewards among the target intersection and its neighbors within a city, and the Upper Transformer learns the general decision trajectories across different cities. This dual-level approach bolsters the model's robust generalization and transferability. Notably, when directly transferring to unseen scenarios, ours surpasses all baseline methods with +7.91% on average, and even +16.3% in some cases, yielding the best results.