# Attention Models in Graphs: A Survey

arXiv: https://arxiv.org/abs/1807.07984 · HTML: https://mlsanity.com/paper/1807.07984/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jul 2018.

## Built on by (most cited first, 30 of 41)

- [How Attentive are Graph Attention Networks?](https://mlsanity.com/paper/2105.14491/) (2021)
- [Learning Dynamic Graph Representation of Brain Connectome with Spatio-Temporal Attention](https://mlsanity.com/paper/2105.13495/) (2021)
- [A Review on Graph Neural Network Methods in Financial Applications](https://mlsanity.com/paper/2111.15367/) (2021)
- [A Survey on Graph Structure Learning: Progress and Opportunities](https://mlsanity.com/paper/2103.03036/) (2021)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [Model Stealing Attacks Against Inductive Graph Neural Networks](https://mlsanity.com/paper/2112.08331/) (2021)
- [Visual Attention Methods in Deep Learning: An In-Depth Survey](https://mlsanity.com/paper/2204.07756/) (2022)
- [Attentive Neural Controlled Differential Equations for Time-series Classification and Forecasting](https://mlsanity.com/paper/2109.01876/) (2021)
- [Attention Mechanism in Neural Networks: Where it Comes and Where it Goes](https://mlsanity.com/paper/2204.13154/) (2022)
- [Fairness-Aware Graph Neural Networks: A Survey](https://mlsanity.com/paper/2307.03929/) (2023)
- [Graph Neural Networks Designed for Different Graph Types: A Survey](https://mlsanity.com/paper/2204.03080/) (2022)
- [Global-Local Attention for Emotion Recognition](https://mlsanity.com/paper/2111.04129/) (2021)
- [Bridging the Gap between Spatial and Spectral Domains: A Unified Framework for Graph Neural Networks](https://mlsanity.com/paper/2107.10234/) (2021)
- [ACE-NODE: Attentive Co-Evolving Neural Ordinary Differential Equations](https://mlsanity.com/paper/2105.14953/) (2021)
- [A survey and taxonomy of loss functions in machine learning](https://mlsanity.com/paper/2301.05579/) (2023)
- [WGCN: Graph Convolutional Networks with Weighted Structural Features](https://mlsanity.com/paper/2104.14060/) (2021)
- [Learning Individual Treatment Effects under Heterogeneous Interference in Networks](https://mlsanity.com/paper/2210.14080/) (2022)
- [Learning Combinatorial Node Labeling Algorithms](https://mlsanity.com/paper/2106.03594/) (2021)
- [A Hypergraph Neural Network Framework for Learning Hyperedge-Dependent Node Embeddings](https://mlsanity.com/paper/2212.14077/) (2022)
- [Recommender systems based on graph embedding techniques: A comprehensive review](https://mlsanity.com/paper/2109.09587/) (2021)
- [Text-Based Automatic Personality Prediction Using KGrAt-Net; A Knowledge Graph Attention Network Classifier](https://mlsanity.com/paper/2205.13780/) (2022)
- [Learnable Graph Convolutional Attention Networks](https://mlsanity.com/paper/2211.11853/) (2022)
- [Graph Neural Networks for Traffic Forecasting](https://mlsanity.com/paper/2104.13096/) (2021)
- [Multi-UAV Conflict Resolution with Graph Convolutional Reinforcement Learning](https://mlsanity.com/paper/2111.14598/) (2021)
- [Graph Neural Networks: a bibliometrics overview](https://mlsanity.com/paper/2201.01188/) (2022)
- [Adversarially Regularized Graph Attention Networks for Inductive Learning on Partially Labeled Graphs](https://mlsanity.com/paper/2106.03393/) (2021)
- [MCDS: AI Augmented Workflow Scheduling in Mobile Edge Cloud Computing Systems](https://mlsanity.com/paper/2112.07269/) (2021)
- [Learning Bi-typed Multi-relational Heterogeneous Graph via Dual Hierarchical Attention Networks](https://mlsanity.com/paper/2112.13078/) (2021)
- [DREAM: Adaptive Reinforcement Learning based on Attention Mechanism for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2304.03984/) (2023)
- [A Survey on Image-text Multimodal Models](https://mlsanity.com/paper/2309.15857/) (2023)

## Abstract (arXiv)

Graph-structured data arise naturally in many different application domains. By representing data as graphs, we can capture entities (i.e., nodes) as well as their relationships (i.e., edges) with each other. Many useful insights can be derived from graph-structured data as demonstrated by an ever-growing body of work focused on graph mining. However, in the real-world, graphs can be both large - with many complex patterns - and noisy which can pose a problem for effective graph mining. An effective way to deal with this issue is to incorporate "attention" into graph mining solutions. An attention mechanism allows a method to focus on task-relevant parts of the graph, helping it to make better decisions. In this work, we conduct a comprehensive and focused survey of the literature on the emerging field of graph attention models. We introduce three intuitive taxonomies to group existing work. These are based on problem setting (type of input and output), the type of attention mechanism used, and the task (e.g., graph classification, link prediction, etc.). We motivate our taxonomies through detailed examples and use each to survey competing approaches from a unique standpoint. Finally, we highlight several challenges in the area and discuss promising directions for future work.