mlsanityData to 14 Oct 2026

Graph Neural Networks with Learnable Structural and Positional Representations

arXiv 2110.07875 ↗First posted 15 Oct 2021cs.LGcited by 333

Not on the map. Why?

Abstract (arXiv)

Graph neural networks (GNNs) have become the standard learning architectures for graphs. GNNs have been applied to numerous domains ranging from quantum chemistry, recommender systems to knowledge graphs and natural language processing. A major issue with arbitrary graphs is the absence of canonical positional information of nodes, which decreases the representation power of GNNs to distinguish e.g. isomorphic nodes and other graph symmetries. An approach to tackle this issue is to introduce Positional Encoding (PE) of nodes, and inject it into the input layer, like in Transformers. Possible graph PE are Laplacian eigenvectors. In this work, we propose to decouple structural and positional representations to make easy for the network to learn these two essential properties. We introduce a novel generic architecture which we call LSPE (Learnable Structural and Positional Encodings). We investigate several sparse and fully-connected (Transformer-like) GNNs, and observe a performance increase for molecular datasets, from 1.79% up to 64.14% when considering learnable PE for both GNN classes.

Key facts. Cited by 333 papers in the map (data through 14 Oct 2026). Builds on 50 of its 74 references that are in the map. First posted on arXiv 15 Oct 2021.

Builds on

50 of 74 references.

References · 50

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  4. 2017Graph Attention Networkscited by 8,923
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  6. 2017Inductive Representation Learning on Large Graphscited by 6,320
  7. 2018How Powerful are Graph Neural Networks?cited by 4,393
  8. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  9. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  10. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,089
  11. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,576
  12. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,051
38 more
  1. 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 881
  2. 2020TUDataset: A collection of benchmark datasets for learning with graphscited by 728
  3. 2020Benchmarking Graph Neural Networkscited by 680
  4. 2020A Generalization of Transformer Networks to Graphscited by 661
  5. 2018Junction Tree Variational Autoencoder for Molecular Graph Generationcited by 652
  6. 2020On the Bottleneck of Graph Neural Networks and its Practical Implicationscited by 631
  7. 2018Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizoncited by 623
  8. 2019DeepGCNs: Can GCNs Go as Deep as CNNs?cited by 527
  9. 2020Principal Neighbourhood Aggregation for Graph Netscited by 478
  10. 2021Rethinking Graph Transformers with Spectral Attentioncited by 473
  11. 2016Geometric deep learning on graphs and manifolds using mixture model CNNscited by 468
  12. 2019Provably Powerful Graph Networkscited by 389
  13. 2016Learning Multiagent Communication with Backpropagationcited by 366
  14. 2017Residual Gated Graph ConvNetscited by 346
  15. 2018Invariant and Equivariant Graph Networkscited by 346
  16. 2021Do Transformers Really Perform Bad for Graph Representation?cited by 320
  17. 2020Improving Graph Neural Network Expressivity via Subgraph Isomorphism Countingcited by 313
  18. 2021Combinatorial optimization and reasoning with graph neural networkscited by 268
  19. 2020DeeperGCN: All You Need to Train Deeper GCNscited by 250
  20. 2019Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networkscited by 242
  21. 2021Weisfeiler and Lehman Go Cellular: CW Networkscited by 218
  22. 2019Position-aware Graph Neural Networkscited by 203
  23. 2019What graph neural networks cannot learn: depth vs widthcited by 201
  24. 2019On the equivalence between graph isomorphism testing and function approximation with GNNscited by 172
  25. 2020How Much Position Information Do Convolutional Neural Networks Encode?cited by 172
  26. 2019Relational Pooling for Graph Representationscited by 150
  27. 2021GraphiT: Encoding Graph Structure in Transformerscited by 147
  28. 2020Directional Graph Networkscited by 133
  29. 2019Fake News Detection on Social Media using Geometric Deep Learningcited by 118
  30. 2020A Survey on The Expressive Power of Graph Neural Networkscited by 111
  31. 2021Position Information in Transformers: An Overviewcited by 103
  32. 2020Expressive Power of Invariant and Equivariant Graph Neural Networkscited by 100
  33. 2020PinnerSage: Multi-Modal User Embedding Framework for Recommendations at Pinterestcited by 75
  34. 2019Approximation Ratios of Graph Neural Networks for Combinatorial Problemscited by 69
  35. 2020Memory-Based Graph Networkscited by 56
  36. 2019Learning Symbolic Physics with Graph Networkscited by 53
  37. 2021On Positional and Structural Node Features for Graph Neural Networks on Non-attributed Graphscited by 49
  38. 2020Utilising Graph Machine Learning within Drug Discovery and Developmentcited by 26

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 45; 2023: 62; 2024: 92; 2025: 85; 2026: 49.

333 papers.

Sort:

By area:

  1. 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 713
  2. 2022Long Range Graph Benchmarkcited by 263
  3. 2022Structure-Aware Transformer for Graph Representation Learningcited by 194
  4. 2022Sign and Basis Invariant Networks for Spectral Graph Representation Learningcited by 179
  5. 2023Exphormer: Sparse Transformers for Graphscited by 170
  6. 2023Graph Inductive Biases in Transformers without Message Passingcited by 153
  7. 2022Understanding and Extending Subgraph GNNs by Rethinking Their Symmetriescited by 112
  8. 2023GraphText: Graph Reasoning in Text Spacecited by 111
  9. 2022Equivariant and Stable Positional Encoding for More Powerful Graph Neural Networkscited by 94
  10. 2022Brain Network Transformercited by 94
  11. 2022A Generalization of ViT/MLP-Mixer to Graphscited by 93
  12. 2022Federated Learning on Non-IID Graphs via Structural Knowledge Sharingcited by 88
38 more
  1. 2023Where Did the Gap Go? Reassessing the Long-Range Graph Benchmarkcited by 85
  2. 2023Attending to Graph Transformerscited by 83
  3. 2023DRew: Dynamically Rewired Message Passing with Delaycited by 82
  4. 2023Ordered GNN: Ordering Message Passing to Deal with Heterophily and Over-smoothingcited by 67
  5. 2024Polynormer: Polynomial-Expressive Graph Transformer in Linear Timecited by 65
  6. 2022Learning to Reconstruct Missing Data from Spatiotemporal Graphs with Sparse Observationscited by 64
  7. 2024GraphCLIP: Enhancing Transferability in Graph Foundation Models for Text-Attributed Graphscited by 58
  8. 2022Beyond Smoothing: Unsupervised Graph Representation Learning with Edge Heterophily Discriminatingcited by 57
  9. 2022Boosting the Cycle Counting Power of Graph Neural Networks with I2-GNNscited by 56
  10. 2025Position: Graph Learning Will Lose Relevance Due To Poor Benchmarkscited by 53
  11. 2022Message passing all the way upcited by 47
  12. 2022GOOD-D: On Unsupervised Graph Out-Of-Distribution Detectioncited by 47
  13. 2023Virtual Node Tuning for Few-shot Node Classificationcited by 46
  14. 2022GraphGDP: Generative Diffusion Processes for Permutation Invariant Graph Generationcited by 43
  15. 2022Tree Mover's Distance: Bridging Graph Metrics and Stability of Graph Neural Networkscited by 41
  16. 2023Towards Foundational Models for Molecular Learning on Large-Scale Multi-Task Datasetscited by 39
  17. 2024Beyond Weisfeiler-Lehman: A Quantitative Framework for GNN Expressivenesscited by 39
  18. 2022OOD Link Prediction Generalization Capabilities of Message-Passing GNNs in Larger Test Graphscited by 38
  19. 2024On the Scalability of GNNs for Molecular Graphscited by 36
  20. 2024Residual Connections and Normalization Can Provably Prevent Oversmoothing in GNNscited by 34
  21. 2025Relational Graph Transformercited by 34
  22. 2023Graph Positional and Structural Encodercited by 33
  23. 2023On the Stability of Expressive Positional Encodings for Graphscited by 33
  24. 2023Networked Time Series Imputation via Position-aware Graph Enhanced Variational Autoencoderscited by 32
  25. 2024Cometh: A continuous-time discrete-state graph diffusion modelcited by 32
  26. 2022Generalised Implicit Neural Representationscited by 31
  27. 2023Graph Positional Encoding via Random Feature Propagationcited by 31
  28. 2023Learning Domain-Independent Heuristics for Grounded and Lifted Planningcited by 31
  29. 2022GPS++: An Optimised Hybrid MPNN/Transformer for Molecular Property Predictioncited by 28
  30. 2023Provably Powerful Graph Neural Networks for Directed Multigraphscited by 27
  31. 2024A Canonicalization Perspective on Invariant and Equivariant Learningcited by 27
  32. 2023A Survey on Spectral Graph Neural Networkscited by 26
  33. 2023Probabilistically Rewired Message-Passing Neural Networkscited by 26
  34. 2025Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellencecited by 26
  35. 2024ODIN: A Single Model for 2D and 3D Segmentationcited by 25
  36. 2024Unifying Generation and Prediction on Graphs with Latent Graph Diffusioncited by 25
  37. 2025Learning Efficient Positional Encodings with Graph Neural Networkscited by 24
  38. 2022Make Heterophily Graphs Better Fit GNN: A Graph Rewiring Approachcited by 23

50 most cited of 333 All 333 (JSON)