# Delayed Propagation Transformer: A Universal Computation Engine towards Practical Control in Cyber-Physical Systems

arXiv: https://arxiv.org/abs/2110.15926 · HTML: https://mlsanity.com/paper/2110.15926/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 54 references that are in the map.
- First posted on arXiv 29 Oct 2021.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [Efficient Transformers: A Survey](https://mlsanity.com/paper/2009.06732/) (2020)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [Universal Transformers](https://mlsanity.com/paper/1807.03819/) (2018)
- [Transformer in Transformer](https://mlsanity.com/paper/2103.00112/) (2021)
- [On the Bottleneck of Graph Neural Networks and its Practical Implications](https://mlsanity.com/paper/2006.05205/) (2020)
- [Graph Transformer Networks](https://mlsanity.com/paper/1911.06455/) (2019)
- [AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting](https://mlsanity.com/paper/2103.14023/) (2021)
- [StarCraft II: A New Challenge for Reinforcement Learning](https://mlsanity.com/paper/1708.04782/) (2017)
- [Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control](https://mlsanity.com/paper/1903.04527/) (2019)
- [CoLight: Learning Network-level Cooperation for Traffic Signal Control](https://mlsanity.com/paper/1905.05717/) (2019)
- [CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario](https://mlsanity.com/paper/1905.05217/) (2019)
- [A Survey of Deep Learning Applications to Autonomous Vehicle Control](https://mlsanity.com/paper/1912.10773/) (2019)
- [ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositing](https://mlsanity.com/paper/1803.01837/) (2018)
- [Graph Transformer for Graph-to-Sequence Learning](https://mlsanity.com/paper/1911.07470/) (2019)
- [A Survey on Traffic Signal Control Methods](https://mlsanity.com/paper/1904.08117/) (2019)
- [Directed Graph Convolutional Network](https://mlsanity.com/paper/2004.13970/) (2020)
- [Directed Acyclic Graph Neural Networks](https://mlsanity.com/paper/2101.07965/) (2021)
- [AttendLight: Universal Attention-Based Reinforcement Learning Model for Traffic Signal Control](https://mlsanity.com/paper/2010.05772/) (2020)
- [Multi-agent Reinforcement Learning for Networked System Control](https://mlsanity.com/paper/2004.01339/) (2020)
- [Deep Reinforcement Learning for Traffic Light Control in Vehicular Networks](https://mlsanity.com/paper/1803.11115/) (2018)
- [Using a Deep Reinforcement Learning Agent for Traffic Signal Control](https://mlsanity.com/paper/1611.01142/) (2016)
- [Neurosymbolic Transformers for Multi-Agent Communication](https://mlsanity.com/paper/2101.03238/) (2021)
- [Neuro-adaptive Cooperative Tracking Control with Prescribed Performance of Unknown Higher-order Nonlinear Multi-agent Systems](https://mlsanity.com/paper/1807.01165/) (2018)

## Built on by (most cited first, 3 of 3)

- [Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practice](https://mlsanity.com/paper/2203.05962/) (2022)
- [M3ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-design](https://mlsanity.com/paper/2210.14793/) (2022)
- [Symbolic Learning to Optimize: Towards Interpretability and Scalability](https://mlsanity.com/paper/2203.06578/) (2022)

## Abstract (arXiv)

Multi-agent control is a central theme in the Cyber-Physical Systems (CPS). However, current control methods either receive non-Markovian states due to insufficient sensing and decentralized design, or suffer from poor convergence. This paper presents the Delayed Propagation Transformer (DePT), a new transformer-based model that specializes in the global modeling of CPS while taking into account the immutable constraints from the physical world. DePT induces a cone-shaped spatial-temporal attention prior, which injects the information propagation and aggregation principles and enables a global view. With physical constraint inductive bias baked into its design, our DePT is ready to plug and play for a broad class of multi-agent systems. The experimental results on one of the most challenging CPS -- network-scale traffic signal control system in the open world -- show that our model outperformed the state-of-the-art expert methods on synthetic and real-world datasets. Our codes are released at: https://github.com/VITA-Group/DePT.