mlsanityData to 7 Oct 2026

Learning to Pool in Graph Neural Networks for Extrapolation

arXiv 2106.06210 ↗First posted 11 Jun 2021cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Graph neural networks (GNNs) are one of the most popular approaches to using deep learning on graph-structured data, and they have shown state-of-the-art performances on a variety of tasks. However, according to a recent study, a careful choice of pooling functions, which are used for the aggregation and readout operations in GNNs, is crucial for enabling GNNs to extrapolate. Without proper choices of pooling functions, which varies across tasks, GNNs completely fail to generalize to out-of-distribution data, while the number of possible choices grows exponentially with the number of layers. In this paper, we present GNP, a Lp norm-like pooling function that is trainable end-to-end for any given task. Notably, GNP generalizes most of the widely-used pooling functions. We verify experimentally that simply using GNP for every aggregation and readout operation enables GNNs to extrapolate well on many node-level, graph-level, and set-related tasks; and GNP sometimes performs even better than the best-performing choices among existing pooling functions.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 27 of its 49 references that are in the map. First posted on arXiv 11 Jun 2021.

Builds on

27 of 49 references.

References · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2017Graph Attention Networkscited by 8,854
  5. 2017Inductive Representation Learning on Large Graphscited by 6,284
  6. 2018How Powerful are Graph Neural Networks?cited by 4,355
  7. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  8. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
  9. 2018Hierarchical Graph Representation Learning with Differentiable Poolingcited by 786
  10. 2020TUDataset: A collection of benchmark datasets for learning with graphscited by 718
  11. 2019Graph U-Netscited by 489
  12. 2020Inductive Representation Learning on Temporal Graphscited by 454
15 more
  1. 2019Self-Attention Graph Poolingcited by 421
  2. 2019Provably Powerful Graph Networkscited by 385
  3. 2020DeeperGCN: All You Need to Train Deeper GCNscited by 249
  4. 2015Order Matters: Sequence to sequence for setscited by 236
  5. 2020How Neural Networks Extrapolate: From Feedforward to Graph Neural Networkscited by 218
  6. 2018Learning a SAT Solver from Single-Bit Supervisioncited by 198
  7. 2019Graph Neural Tangent Kernel: Fusing Graph Neural Networks with Graph Kernelscited by 160
  8. 2019ASAP: Adaptive Structure Aware Pooling for Learning Hierarchical Graph Representationscited by 156
  9. 2018Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networkscited by 129
  10. 2019Neural Execution of Graph Algorithmscited by 118
  11. 2020Chip Placement with Deep Reinforcement Learningcited by 83
  12. 2018Neural Arithmetic Logic Unitscited by 69
  13. 2018Learning to Solve NP-Complete Problems - A Graph Neural Network for Decision TSPcited by 58
  14. 2015Generalizing Pooling Functions in Convolutional Neural Networks: Mixed, Gated, and Treecited by 36
  15. 2013Learned-Norm Pooling for Deep Feedforward and Recurrent Neural Networkscited by 32

Built on by

Cited by year

Citing papers per year: 2022: 4.

4 papers.

  1. 2022Out-Of-Distribution Generalization on Graphs: A Surveycited by 97
  2. 2022BeGin: Extensive Benchmark Scenarios and An Easy-to-use Framework for Graph Continual Learningcited by 8
  3. 2022Regularized Optimal Transport Layers for Generalized Global Pooling Operationscited by 2
  4. 2022Revisiting Global Pooling through the Lens of Optimal Transportcited by 0