mlsanityData to 7 Oct 2026

Focus Where It Matters: Graph Selective State Focused Attention Networks

arXiv 2410.15849 ↗First posted 21 Oct 2024cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Traditional graph neural networks (GNNs) lack scalability and lose individual node characteristics due to over-smoothing, especially in the case of deeper networks. This results in sub-optimal feature representation, affecting the model's performance on tasks involving dynamically changing graphs. To address this issue, we present Graph Selective States Focused Attention Networks (GSANs) based neural network architecture for graph-structured data. The GSAN is enabled by multi-head masked self-attention (MHMSA) and selective state space modeling (S3M) layers to overcome the limitations of GNNs. In GSAN, the MHMSA allows GSAN to dynamically emphasize crucial node connections, particularly in evolving graph environments. The S3M layer enables the network to adjust dynamically in changing node states and improving predictions of node behavior in varying contexts without needing primary knowledge of the graph structure. Furthermore, the S3M layer enhances the generalization of unseen structures and interprets how node states influence link importance. With this, GSAN effectively outperforms inductive and transductive tasks and overcomes the issues that traditional GNNs experience. To analyze the performance behavior of GSAN, a set of state-of-the-art comparative experiments are conducted on graphs benchmark datasets, including Cora, Citeseer, Pubmed network citation, and protein-protein-interaction datasets, as an outcome, GSAN improved the classification accuracy by 1.56%, 8.94%, 0.37%, and 1.54% on F1-score respectively.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 41 of its 77 references that are in the map. First posted on arXiv 21 Oct 2024.

Builds on

41 of 77 references.

References · 41

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2017Graph Attention Networkscited by 8,854
  3. 2017Inductive Representation Learning on Large Graphscited by 6,284
  4. 2018How Powerful are Graph Neural Networks?cited by 4,355
  5. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  6. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  7. 2016node2vec: Scalable Feature Learning for Networkscited by 1,913
  8. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,571
  9. 2014DeepWalk: Online Learning of Social Representationscited by 1,543
  10. 2019Simplifying Graph Convolutional Networkscited by 1,470
  11. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
  12. 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 987
29 more
  1. 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 970
  2. 2018Representation Learning on Graphs with Jumping Knowledge Networkscited by 875
  3. 2015Gated Graph Sequence Neural Networkscited by 792
  4. 2019Heterogeneous Graph Attention Networkcited by 732
  5. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 679
  6. 2018FastGCN: Fast Learning with Graph Convolutional Networks via Importance Samplingcited by 583
  7. 2019Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networkscited by 540
  8. 2016Geometric deep learning on graphs and manifolds using mixture model CNNscited by 468
  9. 2019Diffusion Improves Graph Learningcited by 414
  10. 2020SIGN: Scalable Inception Graph Neural Networkscited by 283
  11. 2021Inductive Representation Learning in Temporal Networks via Causal Anonymous Walkscited by 188
  12. 2020Machine Learning on Graphs: A Model and Comprehensive Taxonomycited by 156
  13. 2021A Survey of Graph Neural Networks for Recommender Systems: Challenges, Methods, and Directionscited by 144
  14. 2020Foundations and modelling of dynamic networks using Dynamic Graph Neural Networks: A surveycited by 123
  15. 2020Understanding and Resolving Performance Degradation in Graph Convolutional Networkscited by 53
  16. 2022Federated Graph Neural Networks: Overview, Techniques and Challengescited by 52
  17. 2024A Comprehensive Survey of Dynamic Graph Neural Networks: Models, Frameworks, Benchmarks, Experiments and Challengescited by 29
  18. 2021SPAGAN: Shortest Path Graph Attention Networkcited by 28
  19. 2022Challenges and Opportunities in Deep Reinforcement Learning with Graph Neural Networks: A Comprehensive review of Algorithms and Applicationscited by 27
  20. 2024STG-Mamba: Spatial-Temporal Graph Learning via Selective State Space Modelcited by 24
  21. 2018Predicting citation counts based on deep neural network learning techniquescited by 21
  22. 2023Node-wise Diffusion for Scalable Graph Learningcited by 17
  23. 2022InducT-GCN: Inductive Graph Convolutional Networks for Text Classificationcited by 15
  24. 2023Dynamic Graph Representation Learning with Neural Networks: A Surveycited by 15
  25. 2024Conformal Inductive Graph Neural Networkscited by 9
  26. 2023Bi-Level Attention Graph Neural Networkscited by 6
  27. 2023Union Subgraph Neural Networkscited by 6
  28. 2024Hierarchical Attention Models for Multi-Relational Graphscited by 2
  29. 2021Adaptive Multi-layer Contrastive Graph Neural Networkscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Attention Beyond Neighborhoods: Reviving Transformer for Graph Clustering
  2. 2020Multi-hop Attention Graph Neural Network
  3. 2024Diffusing to the Top: Boost Graph Neural Networks with Minimal Hyperparameter Tuning
  4. 2022A Generic Shared Attention Mechanism for Various Backbone Neural Networks
  5. 2024What Can We Learn from State Space Models for Machine Learning on Graphs?
  6. 2025Self-Adaptive Graph Mixture of Models
  7. 2024SGFormer: Single-Layer Graph Transformers with Approximation-Free Linear Complexity
  8. 2025Hypergraph Neural Network with State Space Models for Node Classification