# Breaking the Expressive Bottlenecks of Graph Neural Networks

arXiv: https://arxiv.org/abs/2012.07219 · HTML: https://mlsanity.com/paper/2012.07219/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Dec 2020.

## Built on by (most cited first, 6 of 6)

- [Do Transformers Really Perform Bad for Graph Representation?](https://mlsanity.com/paper/2106.05234/) (2021)
- [First Place Solution of KDD Cup 2021 & OGB Large-Scale Challenge Graph Prediction Track](https://mlsanity.com/paper/2106.08279/) (2021)
- [Towards Better Graph Representation Learning with Parameterized Decomposition & Filtering](https://mlsanity.com/paper/2305.06102/) (2023)
- [Graph Property Prediction on Open Graph Benchmark: A Winning Solution by Graph Neural Architecture Search](https://mlsanity.com/paper/2207.06027/) (2022)
- [Group Invariant Global Pooling](https://mlsanity.com/paper/2305.19207/) (2023)
- [Graph Neural Networks-based Hybrid Framework For Predicting Particle Crushing Strength](https://mlsanity.com/paper/2307.13909/) (2023)

## Abstract (arXiv)

Recently, the Weisfeiler-Lehman (WL) graph isomorphism test was used to measure the expressiveness of graph neural networks (GNNs), showing that the neighborhood aggregation GNNs were at most as powerful as 1-WL test in distinguishing graph structures. There were also improvements proposed in analogy to k-WL test (k>1). However, the aggregators in these GNNs are far from injective as required by the WL test, and suffer from weak distinguishing strength, making it become expressive bottlenecks. In this paper, we improve the expressiveness by exploring powerful aggregators. We reformulate aggregation with the corresponding aggregation coefficient matrix, and then systematically analyze the requirements of the aggregation coefficient matrix for building more powerful aggregators and even injective aggregators. It can also be viewed as the strategy for preserving the rank of hidden features, and implies that basic aggregators correspond to a special case of low-rank transformations. We also show the necessity of applying nonlinear units ahead of aggregation, which is different from most aggregation-based GNNs. Based on our theoretical analysis, we develop two GNN layers, ExpandingConv and CombConv. Experimental results show that our models significantly boost performance, especially for large and densely connected graphs.