# DropMessage: Unifying Random Dropping for Graph Neural Networks

arXiv: https://arxiv.org/abs/2204.10037 · HTML: https://mlsanity.com/paper/2204.10037/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 62 of its 98 references that are in the map.
- First posted on arXiv 21 Apr 2022.

## Builds on (most cited first, 30 of 62)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [Variational Graph Auto-Encoders](https://mlsanity.com/paper/1611.07308/) (2016)
- [Certified Adversarial Robustness via Randomized Smoothing](https://mlsanity.com/paper/1902.02918/) (2019)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Graph Convolutional Neural Networks for Web-Scale Recommender Systems](https://mlsanity.com/paper/1806.01973/) (2018)
- [Deep Graph Infomax](https://mlsanity.com/paper/1809.10341/) (2018)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning](https://mlsanity.com/paper/1801.07606/) (2018)
- [MoleculeNet: A Benchmark for Molecular Machine Learning](https://mlsanity.com/paper/1703.00564/) (2017)
- [Predict then Propagate: Graph Neural Networks meet Personalized PageRank](https://mlsanity.com/paper/1810.05997/) (2018)
- [Representation Learning on Graphs with Jumping Knowledge Networks](https://mlsanity.com/paper/1806.03536/) (2018)
- [Simple and Deep Graph Convolutional Networks](https://mlsanity.com/paper/2007.02133/) (2020)
- [How Attentive are Graph Attention Networks?](https://mlsanity.com/paper/2105.14491/) (2021)
- [Geom-GCN: Geometric Graph Convolutional Networks](https://mlsanity.com/paper/2002.05287/) (2020)
- [Gated Graph Sequence Neural Networks](https://mlsanity.com/paper/1511.05493/) (2015)
- [DropEdge: Towards Deep Graph Convolutional Networks on Node Classification](https://mlsanity.com/paper/1907.10903/) (2019)
- [GraphSAINT: Graph Sampling Based Inductive Learning Method](https://mlsanity.com/paper/1907.04931/) (2019)
- [FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling](https://mlsanity.com/paper/1801.10247/) (2018)

## Built on by (most cited first, 29 of 29)

- [A Comprehensive Survey on Data Augmentation](https://mlsanity.com/paper/2405.09591/) (2024)
- [Beyond Over-smoothing: Uncovering the Trainability Challenges in Deep Graph Neural Networks](https://mlsanity.com/paper/2408.03669/) (2024)
- [ADEdgeDrop: Adversarial Edge Dropping for Robust Graph Neural Networks](https://mlsanity.com/paper/2403.09171/) (2024)
- [Exploring Correlations of Self-Supervised Tasks for Graphs](https://mlsanity.com/paper/2405.04245/) (2024)
- [Sparse Covariance Neural Networks](https://mlsanity.com/paper/2410.01669/) (2024)
- [Efficient Topology-aware Data Augmentation for High-Degree Graph Neural Networks](https://mlsanity.com/paper/2406.05482/) (2024)
- [IntraMix: Intra-Class Mixup Generation for Accurate Labels and Neighbors](https://mlsanity.com/paper/2405.00957/) (2024)
- [Oversmoothing Alleviation in Graph Neural Networks: A Survey and Unified View](https://mlsanity.com/paper/2405.01663/) (2024)
- [xAI-Drop: Don't Use What You Cannot Explain](https://mlsanity.com/paper/2407.20067/) (2024)
- [GroupFace: Imbalanced Age Estimation Based on Multi-hop Attention Graph Convolutional Network and Group-aware Margin Optimization](https://mlsanity.com/paper/2412.11450/) (2024)
- [Aggregation Buffer: Revisiting DropEdge with a New Parameter Block](https://mlsanity.com/paper/2505.20840/) (2025)
- [Unifying Graph Contrastive Learning via Graph Message Augmentation](https://mlsanity.com/paper/2401.03638/) (2024)
- [Unifews: You Need Fewer Operations for Efficient Graph Neural Networks](https://mlsanity.com/paper/2403.13268/) (2024)
- [Structure-Aware Consensus Network on Graphs with Few Labeled Nodes](https://mlsanity.com/paper/2407.02188/) (2024)
- [TSC: A Simple Two-Sided Constraint against Over-Smoothing](https://mlsanity.com/paper/2408.03152/) (2024)
- [Refining Latent Representations: A Generative SSL Approach for Heterogeneous Graph Learning](https://mlsanity.com/paper/2310.11102/) (2023)
- [Adaptive Multi-Modality Prompt Learning](https://mlsanity.com/paper/2312.00823/) (2023)
- [PepGB: Facilitating peptide drug discovery via graph neural networks](https://mlsanity.com/paper/2401.14665/) (2024)
- [FlexiDrop: Theoretical Insights and Practical Advances in Random Dropout Method on GNNs](https://mlsanity.com/paper/2405.20012/) (2024)
- [FedGrAINS: Personalized SubGraph Federated Learning with Adaptive Neighbor Sampling](https://mlsanity.com/paper/2501.12592/) (2025)
- [Effects of Dropout on Performance in Long-range Graph Learning Tasks](https://mlsanity.com/paper/2502.07364/) (2025)
- [Unifying gradient regularization for Heterogeneous Graph Neural Networks](https://mlsanity.com/paper/2305.15811/) (2023)
- [Exploring Graph Classification Techniques Under Low Data Constraints: A Comprehensive Study](https://mlsanity.com/paper/2311.12737/) (2023)
- [PAC Learnability under Explanation-Preserving Graph Perturbations](https://mlsanity.com/paper/2402.05039/) (2024)
- [P-DROP: Poisson-Based Dropout for Graph Neural Networks](https://mlsanity.com/paper/2505.21783/) (2025)
- [CoATA: Effective Co-Augmentation of Topology and Attribute for Graph Neural Networks](https://mlsanity.com/paper/2506.22299/) (2025)
- [ParaFormer: A Generalized PageRank Graph Transformer for Graph Representation Learning](https://mlsanity.com/paper/2512.14619/) (2025)
- [RADE: Random Add-Drop Edge as a Regularizer](https://mlsanity.com/paper/2606.00757/) (2026)
- [ScaGNN: a Graph Neural Network for Multiple Scattering Simulations](https://mlsanity.com/paper/2609.37509/) (2026)

## Abstract (arXiv)

Graph Neural Networks (GNNs) are powerful tools for graph representation learning. Despite their rapid development, GNNs also face some challenges, such as over-fitting, over-smoothing, and non-robustness. Previous works indicate that these problems can be alleviated by random dropping methods, which integrate augmented data into models by randomly masking parts of the input. However, some open problems of random dropping on GNNs remain to be solved. First, it is challenging to find a universal method that are suitable for all cases considering the divergence of different datasets and models. Second, augmented data introduced to GNNs causes the incomplete coverage of parameters and unstable training process. Third, there is no theoretical analysis on the effectiveness of random dropping methods on GNNs. In this paper, we propose a novel random dropping method called DropMessage, which performs dropping operations directly on the propagated messages during the message-passing process. More importantly, we find that DropMessage provides a unified framework for most existing random dropping methods, based on which we give theoretical analysis of their effectiveness. Furthermore, we elaborate the superiority of DropMessage: it stabilizes the training process by reducing sample variance; it keeps information diversity from the perspective of information theory, enabling it become a theoretical upper bound of other methods. To evaluate our proposed method, we conduct experiments that aims for multiple tasks on five public datasets and two industrial datasets with various backbone models. The experimental results show that DropMessage has the advantages of both effectiveness and generalization, and can significantly alleviate the problems mentioned above.