mlsanityData to 14 Oct 2026

Graph Propagation Transformer for Graph Representation Learning

arXiv 2305.11424 ↗First posted 19 May 2023cs.LG cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

This paper presents a novel transformer architecture for graph representation learning. The core insight of our method is to fully consider the information propagation among nodes and edges in a graph when building the attention module in the transformer blocks. Specifically, we propose a new attention mechanism called Graph Propagation Attention (GPA). It explicitly passes the information among nodes and edges in three ways, i.e. node-to-node, node-to-edge, and edge-to-node, which is essential for learning graph-structured data. On this basis, we design an effective transformer architecture named Graph Propagation Transformer (GPTrans) to further help learn graph data. We verify the performance of GPTrans in a wide range of graph learning experiments on several benchmark datasets. These results show that our method outperforms many state-of-the-art transformer-based graph models with better performance. The code will be released at https://github.com/czczup/GPTrans.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 43 of its 58 references that are in the map. First posted on arXiv 19 May 2023.

Builds on

43 of 58 references.

References · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  6. 2017Graph Attention Networkscited by 8,923
  7. 2017Inductive Representation Learning on Large Graphscited by 6,320
  8. 2016Layer Normalizationcited by 5,669
  9. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  10. 2018How Powerful are Graph Neural Networks?cited by 4,393
  11. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  12. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
31 more
  1. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,089
  2. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  3. 2014DeepWalk: Online Learning of Social Representationscited by 1,547
  4. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,102
  5. 2018Image Transformercited by 772
  6. 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 713
  7. 2020Benchmarking Graph Neural Networkscited by 680
  8. 2020A Generalization of Transformer Networks to Graphscited by 661
  9. 2018FastGCN: Fast Learning with Graph Convolutional Networks via Importance Samplingcited by 584
  10. 2022Vision Transformer Adapter for Dense Predictionscited by 565
  11. 2019Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological Viewcited by 545
  12. 2022InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutionscited by 482
  13. 2020Principal Neighbourhood Aggregation for Graph Netscited by 478
  14. 2021Rethinking Graph Transformers with Spectral Attentioncited by 473
  15. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 447
  16. 2017Residual Gated Graph ConvNetscited by 346
  17. 2021OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphscited by 329
  18. 2020DeeperGCN: All You Need to Train Deeper GCNscited by 250
  19. 2015Deep Convolutional Networks on Graph-Structured Datacited by 246
  20. 2022Pure Transformers are Powerful Graph Learnerscited by 188
  21. 2020Graph-Bert: Only Attention is Needed for Learning Graph Representationscited by 177
  22. 2023DDP: Diffusion Model for Dense Visual Predictioncited by 148
  23. 2020Directional Graph Networkscited by 133
  24. 2021Global Self-Attention as a Replacement for Graph Convolutioncited by 114
  25. 2020Non-Local Graph Neural Networkscited by 91
  26. 2019Graph Transformer for Graph-to-Sequence Learningcited by 82
  27. 2022GRPE: Relative Positional Encoding for Graph Transformercited by 70
  28. 2022Benchmarking Graphormer on Large-Scale Molecular Modeling Datasetscited by 64
  29. 2022DGraph: A Large-Scale Financial Dataset for Graph Anomaly Detectioncited by 63
  30. 2022InternVideo-Ego4D: A Pack of Champion Solutions to Ego4D Challengescited by 48
  31. 2020Graph convolutions that can finally model local structurecited by 31

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 6; 2025: 2.

9 papers.

  1. 2024Spatio-Spectral Graph Neural Networkscited by 17
  2. 2024FlowerFormer: Empowering Neural Architecture Encoding using a Flow-aware Graph Transformercited by 10
  3. 2024Impact of Domain Knowledge and Multi-Modality on Intelligent Molecular Property Prediction: A Systematic Surveycited by 6
  4. 2024Unleashing the Infinity Power of Geometry: A Novel Geometry-Aware Transformer (GOAT) for Whole Slide Histopathology Image Analysiscited by 3
  5. 2025Generalizable Insights for Graph Transformers in Theory and Practicecited by 2
  6. 2025Simple Path Structural Encoding for Graph Transformerscited by 1
  7. 2023Hybrid Focal and Full-Range Attention Based Graph Transformerscited by 0
  8. 2024Dual-perspective Cross Contrastive Learning in Graph Transformerscited by 0
  9. 2024Modeling Text-Label Alignment for Hierarchical Text Classificationcited by 0