# Relation-aware Graph Attention Model With Adaptive Self-adversarial Training

arXiv: https://arxiv.org/abs/2102.07186 · HTML: https://mlsanity.com/paper/2102.07186/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 29 references that are in the map.
- First posted on arXiv 14 Feb 2021.

## Builds on (most cited first, 24 of 24)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Graph Convolutional Neural Networks for Web-Scale Recommender Systems](https://mlsanity.com/paper/1806.01973/) (2018)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Heterogeneous Graph Attention Network](https://mlsanity.com/paper/1903.07293/) (2019)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [Heterogeneous Graph Transformer](https://mlsanity.com/paper/2003.01332/) (2020)
- [Iterative Deep Graph Learning for Graph Neural Networks: Better and Robust Node Embeddings](https://mlsanity.com/paper/2006.13009/) (2020)
- [Representation Learning for Attributed Multiplex Heterogeneous Network](https://mlsanity.com/paper/1905.01669/) (2019)
- [Understanding Negative Sampling in Graph Representation Learning](https://mlsanity.com/paper/2005.09863/) (2020)
- [KBGAN: Adversarial Learning for Knowledge Graph Embeddings](https://mlsanity.com/paper/1711.04071/) (2017)
- [Reinforcement Learning Based Graph-to-Sequence Model for Natural Question Generation](https://mlsanity.com/paper/1908.04942/) (2019)
- [NSCaching: Simple and Efficient Negative Sampling for Knowledge Graph Embedding](https://mlsanity.com/paper/1812.06410/) (2018)
- [Incorporating GAN for Negative Sampling in Knowledge Representation Learning](https://mlsanity.com/paper/1809.11017/) (2018)
- [Toward Subgraph-Guided Knowledge Graph Question Generation with Graph Neural Networks](https://mlsanity.com/paper/2004.06015/) (2020)
- [Heterogeneous Network Representation Learning: A Unified Framework with Survey and Benchmark](https://mlsanity.com/paper/2004.00216/) (2020)
- [On the Equivalence of Holographic and Complex Embeddings for Link Prediction](https://mlsanity.com/paper/1702.05563/) (2017)

## Built on by (most cited first, 12 of 12)

- [Negative Sampling in Knowledge Graph Representation Learning: A Review](https://mlsanity.com/paper/2402.19195/) (2024)
- [Heterogeneous Graph Neural Network with Multi-view Representation Learning](https://mlsanity.com/paper/2108.13650/) (2021)
- [Heterogeneous Graph Contrastive Learning with Meta-path Contexts and Adaptively Weighted Negative Samples](https://mlsanity.com/paper/2212.13847/) (2022)
- [An Embedding-Based Grocery Search Model at Instacart](https://mlsanity.com/paper/2209.05555/) (2022)
- [MHNF: Multi-hop Heterogeneous Neighborhood information Fusion graph representation learning](https://mlsanity.com/paper/2106.09289/) (2021)
- [Graph Feature Preprocessor: Real-time Subgraph-based Feature Extraction for Financial Crime Detection](https://mlsanity.com/paper/2402.08593/) (2024)
- [Document Structure aware Relational Graph Convolutional Networks for Ontology Population](https://mlsanity.com/paper/2104.12950/) (2021)
- [Bayesian Negative Sampling for Recommendation](https://mlsanity.com/paper/2204.06520/) (2022)
- [Energy-based Generative Models for Target-specific Drug Discovery](https://mlsanity.com/paper/2212.02404/) (2022)
- [Attention-based graph neural networks: a survey](https://mlsanity.com/paper/2605.08679/) (2026)
- [Scaling Knowledge Graph Embedding Models](https://mlsanity.com/paper/2201.02791/) (2022)
- [Learning Robust Heterogeneous Graph Representations via Contrastive-Reconstruction under Sparse Semantics](https://mlsanity.com/paper/2506.06682/) (2025)

## Abstract (arXiv)

This paper describes an end-to-end solution for the relationship prediction task in heterogeneous, multi-relational graphs. We particularly address two building blocks in the pipeline, namely heterogeneous graph representation learning and negative sampling. Existing message passing-based graph neural networks use edges either for graph traversal and/or selection of message encoding functions. Ignoring the edge semantics could have severe repercussions on the quality of embeddings, especially when dealing with two nodes having multiple relations. Furthermore, the expressivity of the learned representation depends on the quality of negative samples used during training. Although existing hard negative sampling techniques can identify challenging negative relationships for optimization, new techniques are required to control false negatives during training as false negatives could corrupt the learning process. To address these issues, first, we propose RelGNN -- a message passing-based heterogeneous graph attention model. In particular, RelGNN generates the states of different relations and leverages them along with the node states to weigh the messages. RelGNN also adopts a self-attention mechanism to balance the importance of attribute features and topological features for generating the final entity embeddings. Second, we introduce a parameter-free negative sampling technique -- adaptive self-adversarial (ASA) negative sampling. ASA reduces the false-negative rate by leveraging positive relationships to effectively guide the identification of true negative samples. Our experimental evaluation demonstrates that RelGNN optimized by ASA for relationship prediction improves state-of-the-art performance across established benchmarks as well as on a real industrial dataset.