mlsanityData to 14 Oct 2026

Delayed Propagation Transformer: A Universal Computation Engine towards Practical Control in Cyber-Physical Systems

arXiv 2110.15926 ↗First posted 29 Oct 2021cs.AI cs.SY eess.SYcited by 3

Not on the map. Why?

Abstract (arXiv)

Multi-agent control is a central theme in the Cyber-Physical Systems (CPS). However, current control methods either receive non-Markovian states due to insufficient sensing and decentralized design, or suffer from poor convergence. This paper presents the Delayed Propagation Transformer (DePT), a new transformer-based model that specializes in the global modeling of CPS while taking into account the immutable constraints from the physical world. DePT induces a cone-shaped spatial-temporal attention prior, which injects the information propagation and aggregation principles and enables a global view. With physical constraint inductive bias baked into its design, our DePT is ready to plug and play for a broad class of multi-agent systems. The experimental results on one of the most challenging CPS -- network-scale traffic signal control system in the open world -- show that our model outperformed the state-of-the-art expert methods on synthetic and real-world datasets. Our codes are released at: https://github.com/VITA-Group/DePT.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 29 of its 54 references that are in the map. First posted on arXiv 29 Oct 2021.

Builds on

29 of 54 references.

References · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2020End-to-End Object Detection with Transformerscited by 7,414
  5. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  6. 2021Transformers in Vision: A Surveycited by 1,013
  7. 2020Efficient Transformers: A Surveycited by 790
  8. 2018Image Transformercited by 772
  9. 2018Universal Transformerscited by 647
  10. 2021Transformer in Transformercited by 633
  11. 2020On the Bottleneck of Graph Neural Networks and its Practical Implicationscited by 631
  12. 2019Graph Transformer Networkscited by 523
17 more
  1. 2021AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecastingcited by 331
  2. 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 274
  3. 2019Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Controlcited by 164
  4. 2019CoLight: Learning Network-level Cooperation for Traffic Signal Controlcited by 138
  5. 2019CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenariocited by 115
  6. 2019A Survey of Deep Learning Applications to Autonomous Vehicle Controlcited by 106
  7. 2018ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositingcited by 90
  8. 2019Graph Transformer for Graph-to-Sequence Learningcited by 82
  9. 2019A Survey on Traffic Signal Control Methodscited by 79
  10. 2020Directed Graph Convolutional Networkcited by 73
  11. 2021Directed Acyclic Graph Neural Networkscited by 70
  12. 2020AttendLight: Universal Attention-Based Reinforcement Learning Model for Traffic Signal Controlcited by 62
  13. 2020Multi-agent Reinforcement Learning for Networked System Controlcited by 53
  14. 2018Deep Reinforcement Learning for Traffic Light Control in Vehicular Networkscited by 52
  15. 2016Using a Deep Reinforcement Learning Agent for Traffic Signal Controlcited by 26
  16. 2021Neurosymbolic Transformers for Multi-Agent Communicationcited by 13
  17. 2018Neuro-adaptive Cooperative Tracking Control with Prescribed Performance of Unknown Higher-order Nonlinear Multi-agent Systemscited by 1

Built on by

Cited by year

Citing papers per year: 2022: 3.

3 papers.

  1. 2022Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practicecited by 136
  2. 2022M3ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-designcited by 98
  3. 2022Symbolic Learning to Optimize: Towards Interpretability and Scalabilitycited by 16