mlsanityData to 7 Oct 2026

Supra-Laplacian Encoding for Transformer on Dynamic Graphs

arXiv 2409.17986 ↗First posted 26 Sep 2024cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Fully connected Graph Transformers (GT) have rapidly become prominent in the static graph community as an alternative to Message-Passing models, which suffer from a lack of expressivity, oversquashing, and under-reaching. However, in a dynamic context, by interconnecting all nodes at multiple snapshots with self-attention, GT loose both structural and temporal information. In this work, we introduce Supra-LAplacian encoding for spatio-temporal TransformErs (SLATE), a new spatio-temporal encoding to leverage the GT architecture while keeping spatio-temporal information. Specifically, we transform Discrete Time Dynamic Graphs into multi-layer graphs and take advantage of the spectral properties of their associated supra-Laplacian matrix. Our second contribution explicitly model nodes' pairwise relationships with a cross-attention mechanism, providing an accurate edge representation for dynamic link prediction. SLATE outperforms numerous state-of-the-art methods based on Message-Passing Graph Neural Networks combined with recurrent models (e.g LSTM), and Dynamic Graph Transformers, on 9 datasets. Code is available at: github.com/ykrmm/SLATE.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 45 of its 60 references that are in the map. First posted on arXiv 26 Sep 2024.

Builds on

45 of 60 references.

References · 45

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2017Graph Attention Networkscited by 8,854
  4. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  5. 2016Variational Graph Auto-Encoderscited by 1,373
  6. 2020LightGCN: Simplifying and Powering Graph Convolution Network for Recommendationcited by 1,134
  7. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
  8. 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 878
  9. 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 702
  10. 2020A Generalization of Transformer Networks to Graphscited by 653
  11. 2020On the Bottleneck of Graph Neural Networks and its Practical Implicationscited by 624
  12. 2020Temporal Graph Networks for Deep Learning on Dynamic Graphscited by 521
33 more
  1. 2021Understanding over-squashing and bottlenecks on graphs via curvaturecited by 471
  2. 2021Rethinking Graph Transformers with Spectral Attentioncited by 468
  3. 2020Inductive Representation Learning on Temporal Graphscited by 454
  4. 2019EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphscited by 443
  5. 2019Predicting Dynamic Embedding Trajectory in Temporal Interaction Networkscited by 443
  6. 2021Graph Neural Networks with Learnable Structural and Positional Representationscited by 326
  7. 2021Do Transformers Really Perform Bad for Graph Representation?cited by 318
  8. 2016Structured Sequence Modeling with Graph Convolutional Recurrent Networkscited by 273
  9. 2020Can Graph Neural Networks Count Substructures?cited by 235
  10. 2021Inductive Representation Learning in Temporal Networks via Causal Anonymous Walkscited by 188
  11. 2022Pure Transformers are Powerful Graph Learnerscited by 188
  12. 2023Towards Better Dynamic Graph Learning: New Architecture and Unified Librarycited by 154
  13. 2021GraphiT: Encoding Graph Structure in Transformerscited by 146
  14. 2023Do We Really Need Complicated Model Architectures For Temporal Networks?cited by 141
  15. 2020Directional Graph Networkscited by 132
  16. 2022ROLAND: Graph Learning Framework for Dynamic Graphscited by 131
  17. 2023SGFormer: Simplifying and Empowering Transformers for Large-Graph Representationscited by 131
  18. 2020Foundations and modelling of dynamic networks using Dynamic Graph Neural Networks: A surveycited by 123
  19. 2022Towards Better Evaluation for Dynamic Link Predictioncited by 120
  20. 2021TransMOT: Spatial-Temporal Graph Transformer for Multiple Object Trackingcited by 93
  21. 2013Multilayer Networkscited by 86
  22. 2019Variational Graph Recurrent Neural Networkscited by 86
  23. 2021Discrete-time Temporal Network Embedding via Implicit Hierarchical Learning in Hyperbolic Spacecited by 67
  24. 2021TCL: Transformer-based Dynamic Graph Modelling via Contrastive Learningcited by 65
  25. 2021Anomaly Detection in Dynamic Graphs via Transformercited by 57
  26. 2019Predicting Path Failure In Time-Evolving Graphscited by 54
  27. 2019A Review of Stochastic Block Models and Extensions for Graph Clusteringcited by 46
  28. 2022Equivariant Networks for Crystal Structurescited by 24
  29. 2023Dynamic Graph Representation Learning with Neural Networks: A Surveycited by 15
  30. 2013Clustering on Multi-Layer Graphs via Subspace Analysis on Grassmann Manifoldscited by 13
  31. 2022Time-aware Dynamic Graph Embedding for Asynchronous Structural Evolutioncited by 8
  32. 2023Dynamic Graph Representation Learning for Depression Screening with Transformercited by 4
  33. 2015Multilayer networks: metrics and spectral propertiescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 3.

6 papers.

  1. 2025Unifying Text Semantics and Graph Structures for Temporal Text-attributed Graphs with Large Language Modelscited by 6
  2. 2026Self-Evolving Agents as Dynamic Graph Transformation: A Survey and New Perspectivecited by 4
  3. 2025Understanding and Improving Laplacian Positional Encodings For Temporal GNNscited by 3
  4. 2025MP-GFormer: A 3D-Geometry-Aware Dynamic Graph Transformer Approach for Machining Process Planningcited by 0
  5. 2026SIEFormer: Spectral-Interpretable and -Enhanced Transformer for Generalized Category Discoverycited by 0
  6. 2026RAPNet: Accelerating Algebraic Multigrid with Learned Sparse Correctionscited by 0