# Understanding and Improvement of Adversarial Training for Network Embedding from an Optimization Perspective

arXiv: https://arxiv.org/abs/2105.08007 · HTML: https://mlsanity.com/paper/2105.08007/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 69 references that are in the map.
- First posted on arXiv 17 May 2021.

## Builds on (most cited first, 30 of 33)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [Sharpness-Aware Minimization for Efficiently Improving Generalization](https://mlsanity.com/paper/2010.01412/) (2020)
- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [Revisiting Semi-Supervised Learning with Graph Embeddings](https://mlsanity.com/paper/1603.08861/) (2016)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Adversarial Training for Free!](https://mlsanity.com/paper/1904.12843/) (2019)
- [LINE: Large-scale Information Network Embedding](https://mlsanity.com/paper/1503.03578/) (2015)
- [Adversarial Attack on Graph Structured Data](https://mlsanity.com/paper/1806.02371/) (2018)
- [Topology Attack and Defense for Graph Neural Networks: An Optimization Perspective](https://mlsanity.com/paper/1906.04214/) (2019)
- [GraphGAN: Graph Representation Learning with Generative Adversarial Nets](https://mlsanity.com/paper/1711.08267/) (2017)
- [Adversarial Training for Large Neural Language Models](https://mlsanity.com/paper/2004.08994/) (2020)
- [Graph Adversarial Training: Dynamically Regularizing Based on Graph Structure](https://mlsanity.com/paper/1902.08226/) (2019)
- [Regularizing Neural Networks via Adversarial Model Perturbation](https://mlsanity.com/paper/2010.04925/) (2020)
- [Distributional Smoothing with Virtual Adversarial Training](https://mlsanity.com/paper/1507.00677/) (2015)
- [Domain Adversarial Training for Accented Speech Recognition](https://mlsanity.com/paper/1806.02786/) (2018)
- [Table2Vec: Neural Word and Entity Embeddings for Table Population and Retrieval](https://mlsanity.com/paper/1906.00041/) (2019)
- [Batch Virtual Adversarial Training for Graph Convolutional Networks](https://mlsanity.com/paper/1902.09192/) (2019)
- [Adversarial Training Methods for Network Embedding](https://mlsanity.com/paper/1908.11514/) (2019)
- [GraphDefense: Towards Robust Graph Convolutional Networks](https://mlsanity.com/paper/1911.04429/) (2019)
- [TSSRGCN: Temporal Spectral Spatial Retrieval Graph Convolutional Network for Traffic Flow Forecasting](https://mlsanity.com/paper/2011.14638/) (2020)
- [Adversarial Training of End-to-end Speech Recognition Using a Criticizing Language Model](https://mlsanity.com/paper/1811.00787/) (2018)
- [Virtual Adversarial Training on Graph Convolutional Networks in Node Classification](https://mlsanity.com/paper/1902.11045/) (2019)
- [Tag2Vec: Learning Tag Representations in Tag Networks](https://mlsanity.com/paper/1905.03041/) (2019)
- [Adversarial Defense Framework for Graph Neural Network](https://mlsanity.com/paper/1905.03679/) (2019)

## Built on by (most cited first, 9 of 9)

- [Make Heterophily Graphs Better Fit GNN: A Graph Rewiring Approach](https://mlsanity.com/paper/2209.08264/) (2022)
- [Text-to-Image Generation for Abstract Concepts](https://mlsanity.com/paper/2309.14623/) (2023)
- [FedSI: Federated Subnetwork Inference for Efficient Uncertainty Quantification](https://mlsanity.com/paper/2404.15657/) (2024)
- [Learning Rate Perturbation: A Generic Plugin of Learning Rate Schedule towards Flatter Local Minima](https://mlsanity.com/paper/2208.11873/) (2022)
- [Adversarial Collaborative Filtering for Free](https://mlsanity.com/paper/2308.13541/) (2023)
- [TAROT: A Hierarchical Framework with Multitask Co-Pretraining on Semi-Structured Data towards Effective Person-Job Fit](https://mlsanity.com/paper/2401.07525/) (2024)
- [AdvSGM: Differentially Private Graph Learning via Adversarial Skip-gram Model](https://mlsanity.com/paper/2503.21426/) (2025)
- [Professional Network Matters: Connections Empower Person-Job Fit](https://mlsanity.com/paper/2401.00010/) (2023)
- [Structure-Preference Enabled Graph Embedding Generation under Differential Privacy](https://mlsanity.com/paper/2501.03451/) (2025)

## Abstract (arXiv)

Network Embedding aims to learn a function mapping the nodes to Euclidean space contribute to multiple learning analysis tasks on networks. However, the noisy information behind the real-world networks and the overfitting problem both negatively impact the quality of embedding vectors. To tackle these problems, researchers utilize Adversarial Training for Network Embedding (AdvTNE) and achieve state-of-the-art performance. Unlike the mainstream methods introducing perturbations on the network structure or the data feature, AdvTNE directly perturbs the model parameters, which provides a new chance to understand the mechanism behind. In this paper, we explain AdvTNE theoretically from an optimization perspective. Considering the Power-law property of networks and the optimization objective, we analyze the reason for its excellent results. According to the above analysis, we propose a new activation to enhance the performance of AdvTNE. We conduct extensive experiments on four real networks to validate the effectiveness of our method in node classification and link prediction. The results demonstrate that our method is superior to the state-of-the-art baseline methods.