# Learning Graph Normalization for Graph Neural Networks

arXiv: https://arxiv.org/abs/2009.11746 · HTML: https://mlsanity.com/paper/2009.11746/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Sep 2020.

## Built on by (most cited first, 16 of 16)

- [ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Prediction](https://mlsanity.com/paper/2106.06130/) (2021)
- [Graph neural networks for materials science and chemistry](https://mlsanity.com/paper/2208.09481/) (2022)
- [PingAn-VCGroup's Solution for ICDAR 2021 Competition on Scientific Literature Parsing Task B: Table Recognition to HTML](https://mlsanity.com/paper/2105.01848/) (2021)
- [Improving Expressivity of GNNs with Subgraph-specific Factor Embedded Normalization](https://mlsanity.com/paper/2305.19903/) (2023)
- [From Local to Global: Spectral-Inspired Graph Neural Networks](https://mlsanity.com/paper/2209.12054/) (2022)
- [GRANOLA: Adaptive Normalization for Graph Neural Networks](https://mlsanity.com/paper/2404.13344/) (2024)
- [Foundations and Frontiers of Graph Learning Theory](https://mlsanity.com/paper/2407.03125/) (2024)
- [Taming Gradient Oversmoothing and Expansion in Graph Neural Networks](https://mlsanity.com/paper/2410.04824/) (2024)
- [Graph Neural Networks Need Cluster-Normalize-Activate Modules](https://mlsanity.com/paper/2412.04064/) (2024)
- [Equivariant Neural Network for Factor Graphs](https://mlsanity.com/paper/2109.14218/) (2021)
- [Learning Optimal Topology for Ad-hoc Robot Networks](https://mlsanity.com/paper/2201.12900/) (2022)
- [Improving Peer Assessment with Graph Convolutional Networks](https://mlsanity.com/paper/2111.04466/) (2021)
- [Improved Aggregating and Accelerating Training Methods for Spatial Graph Neural Networks on Fraud Detection](https://mlsanity.com/paper/2202.06580/) (2022)
- [Towards a deeper GCN: Alleviate over-smoothing with iterative training and fine-tuning](https://mlsanity.com/paper/2506.17576/) (2025)
- [Gaussian Rank-Based Neighborhood Degree for Graph Neural Networks in Image Classification](https://mlsanity.com/paper/2605.24367/) (2026)
- [Placing Degree Scales After LayerNorm](https://mlsanity.com/paper/2606.14022/) (2026)

## Abstract (arXiv)

Graph Neural Networks (GNNs) have attracted considerable attention and have emerged as a new promising paradigm to process graph-structured data. GNNs are usually stacked to multiple layers and the node representations in each layer are computed through propagating and aggregating the neighboring node features with respect to the graph. By stacking to multiple layers, GNNs are able to capture the long-range dependencies among the data on the graph and thus bring performance improvements. To train a GNN with multiple layers effectively, some normalization techniques (e.g., node-wise normalization, batch-wise normalization) are necessary. However, the normalization techniques for GNNs are highly task-relevant and different application tasks prefer to different normalization techniques, which is hard to know in advance. To tackle this deficiency, in this paper, we propose to learn graph normalization by optimizing a weighted combination of normalization techniques at four different levels, including node-wise normalization, adjacency-wise normalization, graph-wise normalization, and batch-wise normalization, in which the adjacency-wise normalization and the graph-wise normalization are newly proposed in this paper to take into account the local structure and the global structure on the graph, respectively. By learning the optimal weights, we are able to automatically select a single best or a best combination of multiple normalizations for a specific task. We conduct extensive experiments on benchmark datasets for different tasks, including node classification, link prediction, graph classification and graph regression, and confirm that the learned graph normalization leads to competitive results and that the learned weights suggest the appropriate normalization techniques for the specific task. Source code is released here https://github.com/cyh1112/GraphNormalization.