mlsanityData to 7 Oct 2026

Causal-Based Supervision of Attention in Graph Neural Network: A Better and Simpler Choice towards Powerful Attention

arXiv 2305.13115 ↗First posted 22 May 2023cs.LG cs.AI cs.CYcited by 5

Not on the map. Why?

Abstract (arXiv)

Recent years have witnessed the great potential of attention mechanism in graph representation learning. However, while variants of attention-based GNNs are setting new benchmarks for numerous real-world datasets, recent works have pointed out that their induced attentions are less robust and generalizable against noisy graphs due to lack of direct supervision. In this paper, we present a new framework which utilizes the tool of causality to provide a powerful supervision signal for the learning process of attention functions. Specifically, we estimate the direct causal effect of attention to the final prediction, and then maximize such effect to guide attention attending to more meaningful neighbors. Our method can serve as a plug-and-play module for any canonical attention-based GNNs in an end-to-end fashion. Extensive experiments on a wide range of benchmark datasets illustrated that, by directly supervising attention functions, the model is able to converge faster with a clearer decision boundary, and thus yields better performances.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 45 of its 76 references that are in the map. First posted on arXiv 22 May 2023.

Builds on

45 of 76 references.

References · 45

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2017Graph Attention Networkscited by 8,854
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  5. 2018How Powerful are Graph Neural Networks?cited by 4,355
  6. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  7. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  8. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  9. 2019Simplifying Graph Convolutional Networkscited by 1,470
  10. 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 987
  11. 2018Link Prediction Based on Graph Neural Networkscited by 829
  12. 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 821
33 more
  1. 2021How Attentive are Graph Attention Networks?cited by 818
  2. 2020Adaptive Universal Generalized PageRank Graph Neural Networkcited by 592
  3. 2019Multi-scale Attributed Node Embeddingcited by 543
  4. 2016Interaction Networks for Learning about Objects, Relations and Physicscited by 503
  5. 2020Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classificationcited by 500
  6. 2020Graph Neural Networks in Recommender Systems: A Surveycited by 493
  7. 2021Beyond Low-frequency Information in Graph Convolutional Networkscited by 361
  8. 2017A simple neural network module for relational reasoningcited by 345
  9. 2013Direct and Indirect Effectscited by 276
  10. 2022Finding Global Homophily in Graph Neural Networks When Meeting Heterophilycited by 179
  11. 2019Understanding Attention and Generalization in Graph Neural Networkscited by 169
  12. 2018GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphscited by 164
  13. 2020When Does Self-Supervision Help Graph Convolutional Networks?cited by 121
  14. 2021Causal Attention for Interpretable and Generalizable Graph Classificationcited by 108
  15. 2022How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervisioncited by 102
  16. 2018Attention-based Graph Neural Network for Semi-supervised Learningcited by 100
  17. 2021Breaking the Limit of Graph Neural Networks by Improving the Assortativity of Graphs with Local Mixing Patternscited by 95
  18. 2020Non-Local Graph Neural Networkscited by 91
  19. 2021GBK-GNN: Gated Bi-Kernel Graph Neural Networks for Modeling Both Homophily and Heterophilycited by 84
  20. 2022Graph Attention Multi-Layer Perceptroncited by 82
  21. 2021Graph Posterior Network: Bayesian Predictive Uncertainty for Node Classificationcited by 79
  22. 2021Learning from Counterfactual Links for Link Predictioncited by 69
  23. 2021Scalable and Adaptive Graph Neural Networks with Self-Label-Enhanced trainingcited by 54
  24. 2017VAIN: Attentional Multi-agent Predictive Modelingcited by 51
  25. 2019Sparse Graph Attention Networkscited by 41
  26. 2019Improving Graph Attention Networks with Large Margin-based Constraintscited by 34
  27. 2020Should Graph Convolution Trust Neighbors? A Simple Causal Inference Methodcited by 25
  28. 2021Evolving Attention with Residual Convolutionscited by 20
  29. 2019Graph Representation Learning via Hard and Channel-Wise Attention Networkscited by 18
  30. 2023Robust Mid-Pass Filtering Graph Convolutional Networkscited by 17
  31. 2022ST-ExpertNet: A Deep Expert Framework for Traffic Predictioncited by 10
  32. 2022Easy Begun is Half Done: Spatial-Temporal Graph Modeling with ST-Curriculum Dropoutcited by 7
  33. 2022Causally-guided Regularization of Graph Attention Improves Generalizabilitycited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 3; 2026: 1.

5 papers.

  1. 2023When Graph Neural Network Meets Causality: Opportunities, Methodologies and An Outlookcited by 4
  2. 2025Representation Learning with Mutual Influence of Modalities for Node Classification in Multi-Modal Heterogeneous Networkscited by 1
  3. 2025Causality-Aware Next Location Prediction Framework based on Human Mobility Stratificationcited by 0
  4. 2025Neurocircuitry-Inspired Hierarchical Graph Causal Attention Networks for Explainable Depression Identificationcited by 0
  5. 2026Causal Supervision of Attention for Affective Behaviour Analysiscited by 0