# Investigating Out-of-Distribution Generalization of GNNs: An Architecture Perspective

arXiv: https://arxiv.org/abs/2402.08228 · HTML: https://mlsanity.com/paper/2402.08228/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 73 references that are in the map.
- First posted on arXiv 13 Feb 2024.

## Builds on (most cited first, 30 of 33)

- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Simplifying Graph Convolutional Networks](https://mlsanity.com/paper/1902.07153/) (2019)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Predict then Propagate: Graph Neural Networks meet Personalized PageRank](https://mlsanity.com/paper/1810.05997/) (2018)
- [Adaptive Universal Generalized PageRank Graph Neural Network](https://mlsanity.com/paper/2006.07988/) (2020)
- [Pretrained Transformers Improve Out-of-Distribution Robustness](https://mlsanity.com/paper/2004.06100/) (2020)
- [Neural Bellman-Ford Networks: A General Graph Neural Network Framework for Link Prediction](https://mlsanity.com/paper/2106.06935/) (2021)
- [Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization](https://mlsanity.com/paper/2106.06607/) (2021)
- [Handling Distribution Shifts on Graphs: An Invariance Perspective](https://mlsanity.com/paper/2202.02466/) (2022)
- [Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphs](https://mlsanity.com/paper/2202.05441/) (2022)
- [Understanding The Robustness in Vision Transformers](https://mlsanity.com/paper/2204.12451/) (2022)
- [GOOD: A Graph Out-of-Distribution Benchmark](https://mlsanity.com/paper/2206.08452/) (2022)
- [Causal Attention for Interpretable and Generalizable Graph Classification](https://mlsanity.com/paper/2112.15089/) (2021)
- [Dirichlet Energy Constrained Learning for Deep Graph Neural Networks](https://mlsanity.com/paper/2107.02392/) (2021)
- [Out-Of-Distribution Generalization on Graphs: A Survey](https://mlsanity.com/paper/2202.07987/) (2022)
- [Shift-Robust GNNs: Overcoming the Limitations of Localized Graph Training Data](https://mlsanity.com/paper/2108.01099/) (2021)
- [Invariant Information Bottleneck for Domain Generalization](https://mlsanity.com/paper/2106.06333/) (2021)
- [Delving Deep into the Generalization of Vision Transformers under Distribution Shifts](https://mlsanity.com/paper/2106.07617/) (2021)
- [Empowering Graph Representation Learning with Test-Time Graph Transformation](https://mlsanity.com/paper/2210.03561/) (2022)
- [Graph Trend Filtering Networks for Recommendations](https://mlsanity.com/paper/2108.05552/) (2021)
- [On the Equivalence of Decoupled Graph Convolution Network and Label Propagation](https://mlsanity.com/paper/2010.12408/) (2020)
- [OOD Link Prediction Generalization Capabilities of Message-Passing GNNs in Larger Test Graphs](https://mlsanity.com/paper/2205.15117/) (2022)
- [Mind the Label Shift of Augmentation-based Graph OOD Generalization](https://mlsanity.com/paper/2303.14859/) (2023)
- [Geodesic Graph Neural Network for Efficient Graph Representation Learning](https://mlsanity.com/paper/2210.02636/) (2022)
- [Graph Neural Networks for Multimodal Single-Cell Data Integration](https://mlsanity.com/paper/2203.01884/) (2022)
- [Collaboration-Aware Graph Convolutional Network for Recommender Systems](https://mlsanity.com/paper/2207.06221/) (2022)

## Built on by (most cited first, 13 of 13)

- [A Survey of Deep Graph Learning under Distribution Shifts: from Graph Out-of-Distribution Generalization to Adaptation](https://mlsanity.com/paper/2410.19265/) (2024)
- [Safety in Graph Machine Learning: Threats and Safeguards](https://mlsanity.com/paper/2405.11034/) (2024)
- [USBD: Universal Structural Basis Distillation for Source-Free Graph Domain Adaptation](https://mlsanity.com/paper/2602.08431/) (2026)
- [Toward Robust Non-Transferable Learning: A Survey and Benchmark](https://mlsanity.com/paper/2502.13593/) (2025)
- [Graph Neural Network Generalization with Gaussian Mixture Model Based Augmentation](https://mlsanity.com/paper/2411.08638/) (2024)
- [Invariant Graph Transformer for Out-of-Distribution Generalization](https://mlsanity.com/paper/2508.00304/) (2025)
- [Quantifying Distributional Invariance in Causal Subgraph for IRM-Free Graph Generalization](https://mlsanity.com/paper/2510.20295/) (2025)
- [Exploring Graph-Transformer Out-of-Distribution Generalization Abilities](https://mlsanity.com/paper/2506.20575/) (2025)
- [Learning Invariant Graph Representations Through Redundant Information](https://mlsanity.com/paper/2512.06154/) (2025)
- [Domain Adaptive Unfolded Graph Neural Networks](https://mlsanity.com/paper/2411.13137/) (2024)
- [NodeNAS: Node-Specific Graph Neural Architecture Search for Out-of-Distribution Generalization](https://mlsanity.com/paper/2503.02448/) (2025)
- [Rethinking Graph Generalization through the Lens of Sharpness-Aware Minimization](https://mlsanity.com/paper/2602.08855/) (2026)
- [Temporal Graph Prototype-conditioned Conformal Prediction for Fraud Detection](https://mlsanity.com/paper/2608.15768/) (2026)

## Abstract (arXiv)

Graph neural networks (GNNs) have exhibited remarkable performance under the assumption that test data comes from the same distribution of training data. However, in real-world scenarios, this assumption may not always be valid. Consequently, there is a growing focus on exploring the Out-of-Distribution (OOD) problem in the context of graphs. Most existing efforts have primarily concentrated on improving graph OOD generalization from two model-agnostic perspectives: data-driven methods and strategy-based learning. However, there has been limited attention dedicated to investigating the impact of well-known GNN model architectures on graph OOD generalization, which is orthogonal to existing research. In this work, we provide the first comprehensive investigation of OOD generalization on graphs from an architecture perspective, by examining the common building blocks of modern GNNs. Through extensive experiments, we reveal that both the graph self-attention mechanism and the decoupled architecture contribute positively to graph OOD generalization. In contrast, we observe that the linear classification layer tends to compromise graph OOD generalization capability. Furthermore, we provide in-depth theoretical insights and discussions to underpin these discoveries. These insights have empowered us to develop a novel GNN backbone model, DGAT, designed to harness the robust properties of both graph self-attention mechanism and the decoupled architecture. Extensive experimental results demonstrate the effectiveness of our model under graph OOD, exhibiting substantial and consistent enhancements across various training strategies.