# Robust Optimization as Data Augmentation for Large-scale Graphs

arXiv: https://arxiv.org/abs/2010.09891 · HTML: https://mlsanity.com/paper/2010.09891/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Oct 2020.

## Built on by (most cited first, 30 of 58)

- [NEFTune: Noisy Embeddings Improve Instruction Finetuning](https://mlsanity.com/paper/2310.05914/) (2023)
- [Out-Of-Distribution Generalization on Graphs: A Survey](https://mlsanity.com/paper/2202.07987/) (2022)
- [Graph Data Augmentation for Graph Machine Learning: A Survey](https://mlsanity.com/paper/2202.08871/) (2022)
- [CoCo: A Coupled Contrastive Framework for Unsupervised Domain Adaptive Graph Classification](https://mlsanity.com/paper/2306.04979/) (2023)
- [Unleashing the Power of Graph Data Augmentation on Covariate Distribution Shift](https://mlsanity.com/paper/2211.02843/) (2022)
- [Data-Centric Learning from Unlabeled Graphs with Diffusion Model](https://mlsanity.com/paper/2303.10108/) (2023)
- [GraphMETRO: Mitigating Complex Graph Distribution Shifts via Mixture of Aligned Experts](https://mlsanity.com/paper/2312.04693/) (2023)
- [Dynamic Graph Representation Learning with Neural Networks: A Survey](https://mlsanity.com/paper/2304.05729/) (2023)
- [PTP: Boosting Stability and Performance of Prompt Tuning with Perturbation-Based Regularizer](https://mlsanity.com/paper/2305.02423/) (2023)
- [Towards Data-centric Graph Machine Learning: Review and Outlook](https://mlsanity.com/paper/2309.10979/) (2023)
- [ResearchCodeAgent: An LLM Multi-Agent System for Automated Codification of Research Methodologies](https://mlsanity.com/paper/2504.20117/) (2025)
- [Pre-training with Fractional Denoising to Enhance Molecular Property Prediction](https://mlsanity.com/paper/2407.11086/) (2024)
- [Half-Hop: A graph upsampling approach for slowing down message passing](https://mlsanity.com/paper/2308.09198/) (2023)
- [Adversarial Camouflage for Node Injection Attack on Graphs](https://mlsanity.com/paper/2208.01819/) (2022)
- [Data Augmentation on Graphs: A Technical Survey](https://mlsanity.com/paper/2212.09970/) (2022)
- [Graph Structure and Feature Extrapolation for Out-of-Distribution Generalization](https://mlsanity.com/paper/2306.08076/) (2023)
- [Graph Out-of-Distribution Generalization with Controllable Data Augmentation](https://mlsanity.com/paper/2308.08344/) (2023)
- [Slight Corruption in Pre-training Data Makes Better Diffusion Models](https://mlsanity.com/paper/2405.20494/) (2024)
- [Cross-Domain Graph Data Scaling: A Showcase with Diffusion Models](https://mlsanity.com/paper/2406.01899/) (2024)
- [Multi-task Bioassay Pre-training for Protein-ligand Binding Affinity Prediction](https://mlsanity.com/paper/2306.04886/) (2023)
- [MARIO: Model Agnostic Recipe for Improving OOD Generalization of Graph Contrastive Learning](https://mlsanity.com/paper/2307.13055/) (2023)
- [ADEdgeDrop: Adversarial Edge Dropping for Robust Graph Neural Networks](https://mlsanity.com/paper/2403.09171/) (2024)
- [Counterfactual Data Augmentation with Denoising Diffusion for Graph Anomaly Detection](https://mlsanity.com/paper/2407.02143/) (2024)
- [Towards Better Generalization with Flexible Representation of Multi-Module Graph Neural Networks](https://mlsanity.com/paper/2209.06589/) (2022)
- [Tokenized Graph Transformer with Neighborhood Augmentation for Node Classification in Large Graphs](https://mlsanity.com/paper/2305.12677/) (2023)
- [IDEA: Invariant Defense for Graph Adversarial Robustness](https://mlsanity.com/paper/2305.15792/) (2023)
- [Structural Node Embeddings with Homomorphism Counts](https://mlsanity.com/paper/2308.15283/) (2023)
- [SymNoise: Advancing Language Model Fine-tuning with Symmetric Noise](https://mlsanity.com/paper/2312.01523/) (2023)
- [FraGNNet: A Deep Probabilistic Model for Tandem Mass Spectrum Prediction](https://mlsanity.com/paper/2404.02360/) (2024)
- [IntraMix: Intra-Class Mixup Generation for Accurate Labels and Neighbors](https://mlsanity.com/paper/2405.00957/) (2024)

## Abstract (arXiv)

Data augmentation helps neural networks generalize better by enlarging the training set, but it remains an open question how to effectively augment graph data to enhance the performance of GNNs (Graph Neural Networks). While most existing graph regularizers focus on manipulating graph topological structures by adding/removing edges, we offer a method to augment node features for better performance. We propose FLAG (Free Large-scale Adversarial Augmentation on Graphs), which iteratively augments node features with gradient-based adversarial perturbations during training. By making the model invariant to small fluctuations in input data, our method helps models generalize to out-of-distribution samples and boosts model performance at test time. FLAG is a general-purpose approach for graph data, which universally works in node classification, link prediction, and graph classification tasks. FLAG is also highly flexible and scalable, and is deployable with arbitrary GNN backbones and large-scale datasets. We demonstrate the efficacy and stability of our method through extensive experiments and ablation studies. We also provide intuitive observations for a deeper understanding of our method.