mlsanityData to 14 Oct 2026

Neighbourhood Transformer: Switchable Attention for Monophily-Aware Graph Learning

arXiv 2604.08980 ↗First posted 10 Apr 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Graph neural networks (GNNs) have been widely adopted in engineering applications such as social network analysis, chemical research and computer vision. However, their efficacy is severely compromised by the inherent homophily assumption, which fails to hold for heterophilic graphs where dissimilar nodes are frequently connected. To address this fundamental limitation in graph learning, we first draw inspiration from the recently discovered monophily property of real-world graphs, and propose Neighbourhood Transformers (NT), a novel paradigm that applies self-attention within every local neighbourhood instead of aggregating messages to the central node as in conventional message-passing GNNs. This design makes NT inherently monophily-aware and theoretically guarantees its expressiveness is no weaker than traditional message-passing frameworks. For practical engineering deployment, we further develop a neighbourhood partitioning strategy equipped with switchable attentions, which reduces the space consumption of NT by over 95% and time consumption by up to 92.67%, significantly expanding its applicability to larger graphs. Extensive experiments on 10 real-world datasets (5 heterophilic and 5 homophilic graphs) show that NT outperforms all current state-of-the-art methods on node classification tasks, demonstrating its superior performance and cross-domain adaptability. The full implementation code of this work is publicly available at https://github.com/cf020031308/MoNT to facilitate reproducibility and industrial adoption.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 41 of its 65 references that are in the map. First posted on arXiv 10 Apr 2026.

Builds on

41 of 65 references.

References · 41

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  4. 2017Graph Attention Networkscited by 8,923
  5. 2017Inductive Representation Learning on Large Graphscited by 6,320
  6. 2018How Powerful are Graph Neural Networks?cited by 4,393
  7. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  8. 2020Rethinking Attention with Performerscited by 1,601
  9. 2018Pitfalls of Graph Neural Network Evaluationcited by 1,007
  10. 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 993
  11. 2020Simple and Deep Graph Convolutional Networkscited by 860
  12. 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 826
29 more
  1. 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 713
  2. 2015Image-based Recommendations on Styles and Substitutescited by 706
  3. 2020On the Bottleneck of Graph Neural Networks and its Practical Implicationscited by 631
  4. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
  5. 2020Adaptive Universal Generalized PageRank Graph Neural Networkcited by 595
  6. 2019Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networkscited by 541
  7. 2021Rethinking Graph Transformers with Spectral Attentioncited by 473
  8. 2021Beyond Low-frequency Information in Graph Convolutional Networkscited by 362
  9. 2017Residual Gated Graph ConvNetscited by 346
  10. 2023A critical look at the evaluation of GNNs under heterophily: Are we really making progress?cited by 343
  11. 2021Graph Neural Networks with Learnable Structural and Positional Representationscited by 333
  12. 2020Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networkscited by 260
  13. 2023NodeFormer: A Scalable Graph Structure Learning Transformer for Node Classificationcited by 210
  14. 2020Graph Neural Networks with Heterophilycited by 202
  15. 2022Finding Global Homophily in Graph Neural Networks When Meeting Heterophilycited by 179
  16. 2022Graph Neural Networks for Graphs with Heterophily: A Surveycited by 175
  17. 2023Exphormer: Sparse Transformers for Graphscited by 170
  18. 2023Graph Inductive Biases in Transformers without Message Passingcited by 153
  19. 2023Edge Directionality Improves Learning on Heterophilic Graphscited by 90
  20. 2023DIFFormer: Scalable (Graph) Transformers Induced by Energy Constrained Diffusioncited by 87
  21. 2021GBK-GNN: Gated Bi-Kernel Graph Neural Networks for Modeling Both Homophily and Heterophilycited by 84
  22. 2021New Benchmarks for Learning on Non-Homophilous Graphscited by 81
  23. 2023Ordered GNN: Ordering Message Passing to Deal with Heterophily and Over-smoothingcited by 67
  24. 2024Polynormer: Polynomial-Expressive Graph Transformer in Linear Timecited by 65
  25. 2023Simple and Asymmetric Graph Contrastive Learning without Augmentationscited by 38
  26. 2023Graph Neural Convection-Diffusion with Heterophilycited by 26
  27. 2022High-Order Pooling for Graph Neural Networks with Tensor Decompositioncited by 24
  28. 2015Homophily and missing links in citation networkscited by 21
  29. 2023Graph Contrastive Learning under Heterophily via Graph Filterscited by 5

Built on by

Not cited yet.