mlsanityData to 7 Oct 2026

Adversarial Weight Perturbation Improves Generalization in Graph Neural Networks

arXiv 2212.04983 ↗First posted 9 Dec 2022cs.LGcited by 21

Not on the map. Why?

Abstract (arXiv)

A lot of theoretical and empirical evidence shows that the flatter local minima tend to improve generalization. Adversarial Weight Perturbation (AWP) is an emerging technique to efficiently and effectively find such minima. In AWP we minimize the loss w.r.t. a bounded worst-case perturbation of the model parameters thereby favoring local minima with a small loss in a neighborhood around them. The benefits of AWP, and more generally the connections between flatness and generalization, have been extensively studied for i.i.d. data such as images. In this paper, we extensively study this phenomenon for graph data. Along the way, we first derive a generalization bound for non-i.i.d. node classification tasks. Then we identify a vanishing-gradient issue with all existing formulations of AWP and we propose a new Weighted Truncated AWP (WT-AWP) to alleviate this issue. We show that regularizing graph neural networks with WT-AWP consistently improves both natural and robust generalization across many different graph learning tasks and models.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 30 of its 45 references that are in the map. First posted on arXiv 9 Dec 2022.

Builds on

30 of 45 references.

References · 30

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2017Graph Attention Networkscited by 8,854
  3. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  4. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  5. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  6. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  7. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  8. 2018Pitfalls of Graph Neural Network Evaluationcited by 994
  9. 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 987
  10. 2017Exploring Generalization in Deep Learningcited by 595
  11. 2019Fantastic Generalization Measures and Where to Find Themcited by 507
  12. 2017Sharp Minima Can Generalize For Deep Netscited by 462
18 more
  1. 2018Adversarial Attacks on Neural Networks for Graph Datacited by 395
  2. 2021ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networkscited by 293
  3. 2019Adversarial Attacks on Graph Neural Networks via Meta Learningcited by 261
  4. 2019Topology Attack and Defense for Graph Neural Networks: An Optimization Perspectivecited by 196
  5. 2018Adversarial Attack and Defense on Graph Data: A Surveycited by 166
  6. 2020Node Similarity Preserving Graph Convolutional Networkscited by 112
  7. 2020A PAC-Bayesian Approach to Generalization Bounds for Graph Neural Networkscited by 84
  8. 2020DeepRobust: A PyTorch Library for Adversarial Attacks and Defensescited by 78
  9. 2020Regularizing Neural Networks via Adversarial Model Perturbationcited by 78
  10. 2016Hiding Individuals and Communities in a Social Networkcited by 77
  11. 2021Subgroup Generalization and Fairness of Graph Neural Networkscited by 69
  12. 2021Relating Adversarially Robust Generalization to Flat Minimacited by 57
  13. 2020Efficient Robustness Certificates for Discrete Data: Sparsity-Aware Randomized Smoothing for Graphs, Images and Morecited by 56
  14. 2022Closing the Generalization Gap of Cross-silo Federated Medical Image Segmentationcited by 41
  15. 2022Faster Adaptive Federated Learningcited by 35
  16. 2022RetrievalGuard: Provably Robust 1-Nearest Neighbor Image Retrievalcited by 15
  17. 2022Towards Robust Dataset Learningcited by 9
  18. 2020Detached Error Feedback for Distributed SGD with Random Sparsificationcited by 6

Built on by

Cited by year

Citing papers per year: 2023: 8; 2024: 10; 2025: 3.

21 papers.

Sort:

By area:

  1. 2024A Survey of Graph Neural Networks in Real world: Imbalance, Noise, Privacy and OOD Challengescited by 54
  2. 2023GraphMETRO: Mitigating Complex Graph Distribution Shifts via Mixture of Aligned Expertscited by 19
  3. 2024A Survey of Deep Graph Learning under Distribution Shifts: from Graph Out-of-Distribution Generalization to Adaptationcited by 16
  4. 2024A Watermark for Order-Agnostic Language Modelscited by 14
  5. 2024Safety in Graph Machine Learning: Threats and Safeguardscited by 11
  6. 2024De-mark: Watermark Removal in Large Language Modelscited by 11
  7. 2023Solving a Class of Non-Convex Minimax Optimization in Federated Learningcited by 10
  8. 2024Distortion-free Watermarks are not Truly Distortion-free under Watermark Key Collisionscited by 10
  9. 2023Decentralized Riemannian Algorithm for Nonconvex Minimax Problemscited by 7
  10. 2024SafePowerGraph: Safety-aware Evaluation of Graph Neural Networks for Transmission Power Gridscited by 5
  11. 2024Fast Graph Sharpness-Aware Minimization for Enhancing and Accelerating Few-Shot Node Classificationcited by 4
  12. 2023Enhance Diffusion to Improve Robust Generalizationcited by 3
9 more
  1. 2023Serverless Federated AUPRC Optimization for Multi-Party Collaborative Imbalanced Data Miningcited by 3
  2. 2024Uncertainty Quantification on Graph Learning: A Surveycited by 3
  3. 2023Addressing the Impact of Localized Training Data in Graph Neural Networkscited by 2
  4. 2025Enhancing Distribution and Label Consistency for Graph Out-of-Distribution Generalizationcited by 2
  5. 2023FRGNN: Mitigating the Impact of Distribution Shift on Graph Neural Networks via Test-Time Feature Reconstructioncited by 1
  6. 2023Cooperation or Competition: Avoiding Player Domination for Multi-Target Robustness via Adaptive Budgetscited by 0
  7. 2024RW-NSGCN: A Robust Approach to Structural Attacks via Negative Samplingcited by 0
  8. 2025GCSAM: Gradient Centralized Sharpness Aware Minimizationcited by 0
  9. 2025Leakage-Free Evaluation and Distribution-Robust Spatio-Temporal Graph Learning for Inductive Krigingcited by 0