mlsanityData to 7 Oct 2026

Non-backtracking Graph Neural Networks

arXiv 2310.07430 ↗First posted 11 Oct 2023cs.LG stat.MLcited by 7

Not on the map. Why?

Abstract (arXiv)

The celebrated message-passing updates for graph neural networks allow representing large-scale graphs with local and computationally tractable updates. However, the updates suffer from backtracking, i.e., a message flowing through the same edge twice and revisiting the previously visited node. Since the number of message flows increases exponentially with the number of updates, the redundancy in local updates prevents the graph neural network from accurately recognizing a particular message flow relevant for downstream tasks. In this work, we propose to resolve such a redundancy issue via the non-backtracking graph neural network (NBA-GNN) that updates a message without incorporating the message from the previously visited node. We theoretically investigate how NBA-GNN alleviates the over-squashing of GNNs, and establish a connection between NBA-GNN and the impressive performance of non-backtracking updates for stochastic block model recovery. Furthermore, we empirically verify the effectiveness of our NBA-GNN on the long-range graph benchmark and transductive node classification problems.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 44 of its 70 references that are in the map. First posted on arXiv 11 Oct 2023.

Builds on

44 of 70 references.

References · 44

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2017Graph Attention Networkscited by 8,854
  3. 2017Inductive Representation Learning on Large Graphscited by 6,284
  4. 2018How Powerful are Graph Neural Networks?cited by 4,355
  5. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  6. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  7. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  8. 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 970
  9. 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 821
  10. 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 702
  11. 2020Benchmarking Graph Neural Networkscited by 672
  12. 2020On the Bottleneck of Graph Neural Networks and its Practical Implicationscited by 624
32 more
  1. 2021Understanding over-squashing and bottlenecks on graphs via curvaturecited by 471
  2. 2021Rethinking Graph Transformers with Spectral Attentioncited by 468
  3. 2019MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixingcited by 442
  4. 2019Diffusion Improves Graph Learningcited by 414
  5. 2017Residual Gated Graph ConvNetscited by 341
  6. 2021OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphscited by 328
  7. 2017Community Detection and Stochastic Block Modelscited by 317
  8. 2019Revisiting Graph Neural Networks: All We Have is Low-Pass Filterscited by 260
  9. 2022Long Range Graph Benchmarkcited by 258
  10. 2022Graph Neural Networks for Graphs with Heterophily: A Surveycited by 175
  11. 2023Exphormer: Sparse Transformers for Graphscited by 166
  12. 2023On Over-Squashing in Message Passing Neural Networks: The Impact of Width, Depth, and Topologycited by 163
  13. 2013Consistency of spectral clustering in stochastic block modelscited by 128
  14. 2023Understanding Oversquashing in GNNs through the Lens of Effective Resistancecited by 94
  15. 2022A Generalization of ViT/MLP-Mixer to Graphscited by 93
  16. 2023Where Did the Gap Go? Reassessing the Long-Range Graph Benchmarkcited by 83
  17. 2023DRew: Dynamically Rewired Message Passing with Delaycited by 81
  18. 2013Loopy Belief Propagation for Approximate Inference: An Empirical Studycited by 76
  19. 2013Spectral redemption: clustering sparse networkscited by 54
  20. 2022A Short Tutorial on The Weisfeiler-Lehman Test And Its Variantscited by 52
  21. 2023Path Neural Networks: Expressive and Accurate Graph Neural Networkscited by 32
  22. 2020Co-embedding of Nodes and Edges with Graph Neural Networkscited by 30
  23. 2015Non-backtracking spectrum of random graphs: community detection and non-regular Ramanujan graphscited by 26
  24. 2022Graph Attention Retrospectivecited by 24
  25. 2022Effects of Graph Convolutions in Multi-layer Networkscited by 24
  26. 2015Optimal Cluster Recovery in the Labeled Stochastic Block Modelcited by 18
  27. 2017Supervised Community Detection with Line Graph Neural Networkscited by 18
  28. 2023CIN++: Enhancing Topological Message Passingcited by 14
  29. 2013Spectral community detection in sparse networkscited by 5
  30. 2022Representation Power of Graph Neural Networks: Improved Expressivity via Algebraic Analysiscited by 3
  31. 2017Faster Clustering via Non-Backtracking Random Walkscited by 2
  32. 2018Non-Backtracking Centrality Based Random Walk on Networkscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 1; 2026: 1.

7 papers.

  1. 2023Adaptive Message Passing: A General Framework to Mitigate Oversmoothing, Oversquashing, and Underreachingcited by 23
  2. 2024Probabilistic Graph Rewiring via Virtual Nodescited by 15
  3. 2024Revisiting Random Walks for Learning on Graphscited by 13
  4. 2024PANDA: Expanded Width-Aware Message Passing Beyond Rewiringcited by 10
  5. 2025HollowFlow: Efficient Sample Likelihood Evaluation using Hollow Message Passingcited by 5
  6. 2026From Message-Passing to Linearized Graph Sequence Modelscited by 1
  7. 2024GEN: A Practical Alternative to Graph Transformers for Long-Range Graph Modelingcited by 0