mlsanityData to 14 Oct 2026

Weaver: Kronecker Product Approximations of Spatiotemporal Attention for Traffic Network Forecasting

arXiv 2511.08888 ↗First posted 12 Nov 2025cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Spatiotemporal forecasting on transportation networks is a complex task that requires understanding how traffic nodes interact within a dynamic, evolving system dictated by traffic flow dynamics and social behavioral patterns. The importance of transportation networks and ITS for modern mobility and commerce necessitates forecasting models that are not only accurate but also interpretable, efficient, and robust under structural or temporal perturbations. Recent approaches, particularly Transformer-based architectures, have improved predictive performance but often at the cost of high computational overhead and diminished architectural interpretability. In this work, we introduce Weaver, a novel attention-based model that applies Kronecker product approximations (KPA) to decompose the PN X PN spatiotemporal attention of O(P^2N^2) complexity into local P X P temporal and N X N spatial attention maps. This Kronecker attention map enables our Parallel-Kronecker Matrix-Vector product (P2-KMV) for efficient spatiotemporal message passing with O(P^2N + N^2P) complexity. To capture real-world traffic dynamics, we address the importance of negative edges in modeling traffic behavior by introducing Valence Attention using the continuous Tanimoto coefficient (CTC), which provides properties conducive to precise latent graph generation and training stability. To fully utilize the model's learning capacity, we introduce the Traffic Phase Dictionary for self-conditioning. Evaluations on PEMS-BAY and METR-LA show that Weaver achieves competitive performance across model categories while training more efficiently.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 46 of its 115 references that are in the map. First posted on arXiv 12 Nov 2025.

Builds on

46 of 115 references.

References · 46

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  4. 2016Layer Normalizationcited by 5,669
  5. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  6. 2019Root Mean Square Layer Normalizationcited by 1,480
  7. 2017Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecastingcited by 1,136
  8. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,070
  9. 2019Graph WaveNet for Deep Spatial-Temporal Graph Modelingcited by 829
  10. 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 713
  11. 2016Language Modeling with Gated Convolutional Networkscited by 702
  12. 2014Neural Turing Machinescited by 644
34 more
  1. 2017Self-Normalizing Neural Networkscited by 598
  2. 2016Interaction Networks for Learning about Objects, Relations and Physicscited by 509
  3. 2019GMAN: A Graph Multi-Attention Network for Traffic Predictioncited by 434
  4. 2018T-GCN: A Temporal Graph ConvolutionalNetwork for Traffic Predictioncited by 401
  5. 2016From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classificationcited by 341
  6. 2016Deep Information Propagationcited by 195
  7. 2022Structure-Aware Transformer for Graph Representation Learningcited by 194
  8. 2020Spatial-Temporal Transformer Networks for Traffic Flow Forecastingcited by 180
  9. 2019Sparse Sequence-to-Sequence Modelscited by 170
  10. 2019Adaptively Sparse Transformerscited by 155
  11. 2021GraphiT: Encoding Graph Structure in Transformerscited by 147
  12. 2021PyTorch Geometric Temporal: Spatiotemporal Signal Processing with Neural Machine Learning Modelscited by 118
  13. 2021Position Information in Transformers: An Overviewcited by 103
  14. 2018Towards Sparse Hierarchical Graph Classifierscited by 100
  15. 2018Tropical Geometry of Deep Neural Networkscited by 95
  16. 2024Theory, Analysis, and Best Practices for Sigmoid Self-Attentioncited by 64
  17. 2019Sparse Graph Attention Networkscited by 41
  18. 2018The Sparse Manifold Transformcited by 36
  19. 2022SimA: Simple Softmax-free Attention for Vision Transformerscited by 32
  20. 2023Fast, Differentiable and Sparse Top-k: a Convex Analysis Perspectivecited by 26
  21. 2016Short-term traffic flow forecasting with spatial-temporal correlation in a hybrid deep learning frameworkcited by 23
  22. 2019Optimal Kronecker-Sum Approximation of Real Time Recurrent Learningcited by 22
  23. 2013Covariance Estimation in High Dimensions via Kronecker Product Expansionscited by 16
  24. 2017Kronecker Recurrent Unitscited by 13
  25. 2020Locally Linear Attributes of ReLU Neural Networkscited by 9
  26. 2017Dynamics over Signed Networkscited by 8
  27. 2022Piecewise Linear Neural Networks and Deep Learningcited by 7
  28. 2014Robust Kronecker Product PCA for Spatio-Temporal Covariance Estimationcited by 6
  29. 2025Sigmoid Self-Attention has Lower Sample Complexity than Softmax Self-Attention: A Mixture-of-Experts Perspectivecited by 4
  30. 2013Kronecker Sum Decompositions of Space-Time Datacited by 3
  31. 2020Pooling in Graph Convolutional Neural Networkscited by 3
  32. 2022Scalable Dynamic Mixture Model with Full Covariance for Probabilistic Traffic Forecastingcited by 3
  33. 2022Memory-Augmented Graph Neural Networks: A Brain-Inspired Reviewcited by 2
  34. 2024Traffic Prediction considering Multiple Levels of Spatial-temporal Information: A Multi-scale Graph Wavelet-based Approachcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.