mlsanityData to 7 Oct 2026

Towards Better Generalization with Flexible Representation of Multi-Module Graph Neural Networks

arXiv 2209.06589 ↗First posted 14 Sep 2022cs.LG cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Graph neural networks (GNNs) have become compelling models designed to perform learning and inference on graph-structured data. However, little work has been done to understand the fundamental limitations of GNNs for scaling to larger graphs and generalizing to out-of-distribution (OOD) inputs. In this paper, we use a random graph generator to systematically investigate how the graph size and structural properties affect the predictive performance of GNNs. We present specific evidence that the average node degree is a key feature in determining whether GNNs can generalize to unseen graphs, and that the use of multiple node update functions can improve the generalization performance of GNNs when dealing with graphs of multimodal degree distributions. Accordingly, we propose a multi-module GNN framework that allows the network to adapt flexibly to new graphs by generalizing a single canonical nonlinear transformation over aggregated inputs. Our results show that the multi-module GNNs improve the OOD generalization on a variety of inference tasks in the direction of diverse structural features.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 60 of its 96 references that are in the map. First posted on arXiv 14 Sep 2022.

Builds on

60 of 96 references.

References · 60

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2017Graph Attention Networkscited by 8,854
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2017Inductive Representation Learning on Large Graphscited by 6,284
  5. 2016Layer Normalizationcited by 5,579
  6. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  7. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  8. 2018How Powerful are Graph Neural Networks?cited by 4,355
  9. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  10. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  11. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  12. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
38 more
  1. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  2. 2019Invariant Risk Minimizationcited by 1,832
  3. 2017Deep Setscited by 1,394
  4. 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
  5. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  6. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  7. 2016Geometric deep learning: going beyond Euclidean datacited by 1,042
  8. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  9. 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 878
  10. 2020Learning to Simulate Complex Physics with Graph Networkscited by 867
  11. 2015Gated Graph Sequence Neural Networkscited by 792
  12. 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 751
  13. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
  14. 2020Benchmarking Graph Neural Networkscited by 672
  15. 2020Adaptive Universal Generalized PageRank Graph Neural Networkcited by 592
  16. 2020Principal Neighbourhood Aggregation for Graph Netscited by 477
  17. 2017Representation Learning on Graphs: Methods and Applicationscited by 456
  18. 2015Neural Module Networkscited by 375
  19. 2018Graph networks as learnable physics engines for inference and controlcited by 242
  20. 2021Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalizationcited by 220
  21. 2020Generalization and Representational Limits of Graph Neural Networkscited by 217
  22. 2020Graph Random Neural Network for Semi-Supervised Learning on Graphscited by 213
  23. 2020Data Augmentation for Graph Neural Networkscited by 200
  24. 2018Learning a SAT Solver from Single-Bit Supervisioncited by 198
  25. 2022Discovering Invariant Rationales for Graph Neural Networkscited by 186
  26. 2022Interpretable and Generalizable Graph Learning via Stochastic Attention Mechanismcited by 171
  27. 2022Handling Distribution Shifts on Graphs: An Invariance Perspectivecited by 163
  28. 2022G-Mixup: Graph Data Augmentation for Graph Classificationcited by 145
  29. 2022Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphscited by 144
  30. 2022GOOD: A Graph Out-of-Distribution Benchmarkcited by 140
  31. 2021Weisfeiler and Leman go Machine Learning: The Story so farcited by 121
  32. 2020A Survey on The Expressive Power of Graph Neural Networkscited by 111
  33. 2021Causal Attention for Interpretable and Generalizable Graph Classificationcited by 108
  34. 2022The CLRS Algorithmic Reasoning Benchmarkcited by 103
  35. 2018Janossy Pooling: Learning Deep Permutation-Invariant Functions for Variable-Size Inputscited by 96
  36. 2021Size-Invariant Graph Representations for Graph Classification Extrapolationscited by 82
  37. 2020Graph Structure of Neural Networkscited by 67
  38. 2020Robust Optimization as Data Augmentation for Large-scale Graphscited by 58

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 0; 2025: 0; 2026: 1.

4 papers.

  1. 2023Does Invariant Graph Learning via Environment Augmentation Learn Invariance?cited by 42
  2. 2023Hierarchical Joint Graph Learning and Multivariate Time Series Forecastingcited by 2
  3. 2026Structurally Human, Semantically Biased: Detecting LLM-Generated References with Embeddings and GNNscited by 1
  4. 2023Connecting the Dots: Graph Neural Network Powered Ensemble and Classification of Medical Imagescited by 0