# MLPInit: Embarrassingly Simple GNN Training Acceleration with MLP Initialization

arXiv: https://arxiv.org/abs/2210.00102 · HTML: https://mlsanity.com/paper/2210.00102/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 71 of its 83 references that are in the map.
- First posted on arXiv 30 Sep 2022.

## Builds on (most cited first, 30 of 71)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [Simplifying Graph Convolutional Networks](https://mlsanity.com/paper/1902.07153/) (2019)
- [Variational Graph Auto-Encoders](https://mlsanity.com/paper/1611.07308/) (2016)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [Graph Convolutional Neural Networks for Web-Scale Recommender Systems](https://mlsanity.com/paper/1806.01973/) (2018)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Deep Graph Infomax](https://mlsanity.com/paper/1809.10341/) (2018)
- [Pitfalls of Graph Neural Network Evaluation](https://mlsanity.com/paper/1811.05868/) (2018)
- [Revisiting Semi-Supervised Learning with Graph Embeddings](https://mlsanity.com/paper/1603.08861/) (2016)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [How Attentive are Graph Attention Networks?](https://mlsanity.com/paper/2105.14491/) (2021)
- [Link Prediction Based on Graph Neural Networks](https://mlsanity.com/paper/1802.09691/) (2018)
- [Graph Neural Networks for Social Recommendation](https://mlsanity.com/paper/1902.07243/) (2019)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Contrastive Multi-View Representation Learning on Graphs](https://mlsanity.com/paper/2006.05582/) (2020)
- [GraphSAINT: Graph Sampling Based Inductive Learning Method](https://mlsanity.com/paper/1907.04931/) (2019)
- [Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks](https://mlsanity.com/paper/1905.07953/) (2019)
- [Deep Graph Contrastive Representation Learning](https://mlsanity.com/paper/2006.04131/) (2020)
- [Graph Contrastive Learning with Adaptive Augmentation](https://mlsanity.com/paper/2010.14945/) (2020)
- [Principal Neighbourhood Aggregation for Graph Nets](https://mlsanity.com/paper/2004.05718/) (2020)

## Built on by (most cited first, 30 of 31)

- [SGFormer: Simplifying and Empowering Transformers for Large-Graph Representations](https://mlsanity.com/paper/2306.10759/) (2023)
- [KAGNNs: Kolmogorov-Arnold Networks meet Graph Learning](https://mlsanity.com/paper/2406.18380/) (2024)
- [Graph Neural Networks are Inherently Good Generalizers: Insights by Bridging GNNs and MLPs](https://mlsanity.com/paper/2212.09034/) (2022)
- [LiGNN: Graph Neural Networks at LinkedIn](https://mlsanity.com/paper/2402.11139/) (2024)
- [PREM: A Simple Yet Effective Approach for Node-Level Graph Anomaly Detection](https://mlsanity.com/paper/2310.11676/) (2023)
- [Can LLMs Convert Graphs to Text-Attributed Graphs?](https://mlsanity.com/paper/2412.10136/) (2024)
- [Graph Transformers for Large Graphs](https://mlsanity.com/paper/2312.11109/) (2023)
- [Training MLPs on Graphs without Supervision](https://mlsanity.com/paper/2412.03864/) (2024)
- [CARL-G: Clustering-Accelerated Representation Learning on Graphs](https://mlsanity.com/paper/2306.06936/) (2023)
- [Promoting Fairness in Link Prediction with Graph Enhancement](https://mlsanity.com/paper/2409.08658/) (2024)
- [TinyGraph: Joint Feature and Node Condensation for Graph Neural Networks](https://mlsanity.com/paper/2407.08064/) (2024)
- [SORSA: Singular Values and Orthonormal Regularized Singular Vectors Adaptation of Large Language Models](https://mlsanity.com/paper/2409.00055/) (2024)
- [TINED: GNNs-to-MLPs by Teacher Injection and Dirichlet Energy Distillation](https://mlsanity.com/paper/2412.11180/) (2024)
- [On the Initialization of Graph Neural Networks](https://mlsanity.com/paper/2312.02622/) (2023)
- [Node Duplication Improves Cold-start Link Prediction](https://mlsanity.com/paper/2402.09711/) (2024)
- [Ada-HGNN: Adaptive Sampling for Scalable Hypergraph Neural Networks](https://mlsanity.com/paper/2405.13372/) (2024)
- [A Scalable and Effective Alternative to Graph Transformers](https://mlsanity.com/paper/2406.12059/) (2024)
- [Fast Graph Sharpness-Aware Minimization for Enhancing and Accelerating Few-Shot Node Classification](https://mlsanity.com/paper/2410.16845/) (2024)
- [GraphBridge: Towards Arbitrary Transfer Learning in GNNs](https://mlsanity.com/paper/2502.19252/) (2025)
- [OmniSage: Large Scale, Multi-Entity Heterogeneous Graph Representation Learning](https://mlsanity.com/paper/2504.17811/) (2025)
- [Editable Graph Neural Network for Node Classifications](https://mlsanity.com/paper/2305.15529/) (2023)
- [Contextualizing MLP-Mixers Spatiotemporally for Urban Data Forecast at Scale](https://mlsanity.com/paper/2307.01482/) (2023)
- [Do We Really Need Graph Convolution During Training? Light Post-Training Graph-ODE for Efficient Recommendation](https://mlsanity.com/paper/2407.18910/) (2024)
- [Haste Makes Waste: A Simple Approach for Scaling Graph Neural Networks](https://mlsanity.com/paper/2410.05416/) (2024)
- [Reducing Oversmoothing through Informed Weight Initialization in Graph Neural Networks](https://mlsanity.com/paper/2410.23830/) (2024)
- [Multi-view Graph Condensation via Tensor Decomposition](https://mlsanity.com/paper/2508.14330/) (2025)
- [On the Equivalence of Graph Convolution and Mixup](https://mlsanity.com/paper/2310.00183/) (2023)
- [Unleashing the potential of GNNs via Bi-directional Knowledge Transfer](https://mlsanity.com/paper/2310.17132/) (2023)
- [SimMLP: Training MLPs on Graphs without Supervision](https://mlsanity.com/paper/2402.08918/) (2024)
- [E-CGL: An Efficient Continual Graph Learner](https://mlsanity.com/paper/2408.09350/) (2024)

## Abstract (arXiv)

Training graph neural networks (GNNs) on large graphs is complex and extremely time consuming. This is attributed to overheads caused by sparse matrix multiplication, which are sidestepped when training multi-layer perceptrons (MLPs) with only node features. MLPs, by ignoring graph context, are simple and faster for graph data, however they usually sacrifice prediction accuracy, limiting their applications for graph data. We observe that for most message passing-based GNNs, we can trivially derive an analog MLP (we call this a PeerMLP) with an equivalent weight space, by setting the trainable parameters with the same shapes, making us curious about textbfhow do GNNs using weights from a fully trained PeerMLP perform? Surprisingly, we find that GNNs initialized with such weights significantly outperform their PeerMLPs, motivating us to use PeerMLP training as a precursor, initialization step to GNN training. To this end, we propose an embarrassingly simple, yet hugely effective initialization method for GNN training acceleration, called MLPInit. Our extensive experiments on multiple large-scale graph datasets with diverse GNN architectures validate that MLPInit can accelerate the training of GNNs (up to 33X speedup on OGB-Products) and often improve prediction performance (e.g., up to 7.97% improvement for GraphSAGE across 7 datasets for node classification, and up to 17.81% improvement across 4 datasets for link prediction on metric Hits@10). The code is available at hrefhttps://github.com/snap-research/MLPInit-for-GNNs.