# Learning Conjoint Attentions for Graph Neural Nets

arXiv: https://arxiv.org/abs/2102.03147 · HTML: https://mlsanity.com/paper/2102.03147/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 61 references that are in the map.
- First posted on arXiv 5 Feb 2021.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [Simplifying Graph Convolutional Networks](https://mlsanity.com/paper/1902.07153/) (2019)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [Pitfalls of Graph Neural Network Evaluation](https://mlsanity.com/paper/1811.05868/) (2018)
- [Predict then Propagate: Graph Neural Networks meet Personalized PageRank](https://mlsanity.com/paper/1810.05997/) (2018)
- [Revisiting Semi-Supervised Learning with Graph Embeddings](https://mlsanity.com/paper/1603.08861/) (2016)
- [Representation Learning on Graphs with Jumping Knowledge Networks](https://mlsanity.com/paper/1806.03536/) (2018)
- [Convolutional Networks on Graphs for Learning Molecular Fingerprints](https://mlsanity.com/paper/1509.09292/) (2015)
- [Benchmarking Graph Neural Networks](https://mlsanity.com/paper/2003.00982/) (2020)
- [LINE: Large-scale Information Network Embedding](https://mlsanity.com/paper/1503.03578/) (2015)
- [Principal Neighbourhood Aggregation for Graph Nets](https://mlsanity.com/paper/2004.05718/) (2020)
- [Geometric deep learning on graphs and manifolds using mixture model CNNs](https://mlsanity.com/paper/1611.08402/) (2016)
- [Diffusion Improves Graph Learning](https://mlsanity.com/paper/1911.05485/) (2019)
- [Learning Convolutional Neural Networks for Graphs](https://mlsanity.com/paper/1605.05273/) (2016)
- [Diffusion-Convolutional Neural Networks](https://mlsanity.com/paper/1511.02136/) (2015)
- [Deep Convolutional Networks on Graph-Structured Data](https://mlsanity.com/paper/1506.05163/) (2015)
- [struc2vec: Learning Node Representations from Structural Identity](https://mlsanity.com/paper/1704.03165/) (2017)
- [Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec](https://mlsanity.com/paper/1710.02971/) (2017)

## Built on by (most cited first, 26 of 26)

- [A Survey on Graph Representation Learning Methods](https://mlsanity.com/paper/2204.01855/) (2022)
- [Towards Deep Attention in Graph Neural Networks: Problems and Remedies](https://mlsanity.com/paper/2306.02376/) (2023)
- [Improving Graph Neural Networks with Learnable Propagation Operators](https://mlsanity.com/paper/2210.17224/) (2022)
- [Causality and Independence Enhancement for Biased Node Classification](https://mlsanity.com/paper/2310.09586/) (2023)
- [Free-Rider and Conflict Aware Collaboration Formation for Cross-Silo Federated Learning](https://mlsanity.com/paper/2410.19321/) (2024)
- [A Novel Approach to Temporal QoS Estimation via Extended Kalman Filter-Incorporated Latent Feature Analysis](https://mlsanity.com/paper/2606.23010/) (2026)
- [Tensor Graph Convolutional Network for Dynamic Graph Representation Learning](https://mlsanity.com/paper/2401.07065/) (2024)
- [Not All Neighbors Are Worth Attending to: Graph Selective Attention Networks for Semi-supervised Learning](https://mlsanity.com/paper/2210.07715/) (2022)
- [Generalizing Aggregation Functions in GNNs:High-Capacity GNNs via Nonlinear Neighborhood Aggregators](https://mlsanity.com/paper/2202.09145/) (2022)
- [Spiking GATs: Learning Graph Attentions via Spiking Neural Network](https://mlsanity.com/paper/2209.13539/) (2022)
- [Graph Triple Attention Network: A Decoupled Perspective](https://mlsanity.com/paper/2408.07654/) (2024)
- [Jensen-Shannon Divergence Message-Passing for Rich-Text Graph Representation Learning](https://mlsanity.com/paper/2512.20094/) (2025)
- [A Bird's-Eye Tutorial of Graph Attention Architectures](https://mlsanity.com/paper/2206.02849/) (2022)
- [Neighbor Overlay-Induced Graph Attention Network](https://mlsanity.com/paper/2408.08788/) (2024)
- [Attention-based graph neural networks: a survey](https://mlsanity.com/paper/2605.08679/) (2026)
- [Customizing Graph Neural Networks using Path Reweighting](https://mlsanity.com/paper/2106.10866/) (2021)
- [A Unified Framework for Exploratory Learning-Aided Community Detection Under Topological Uncertainty](https://mlsanity.com/paper/2304.04497/) (2023)
- [IENE: Identifying and Extrapolating the Node Environment for Out-of-Distribution Generalization on Graphs](https://mlsanity.com/paper/2406.00764/) (2024)
- [Spatial-temporal Graph Convolutional Networks with Diversified Transformation for Dynamic Graph Representation Learning](https://mlsanity.com/paper/2408.02704/) (2024)
- [Graph Clustering with Cross-View Feature Propagation](https://mlsanity.com/paper/2408.06029/) (2024)
- [A Novel Spatiotemporal Coupling Graph Convolutional Network](https://mlsanity.com/paper/2408.07087/) (2024)
- [High-order Graph Neural Networks with Common Neighbor Awareness for Link Prediction](https://mlsanity.com/paper/2504.18758/) (2025)
- [A Conjoint Graph Representation Learning Framework for Hypertension Comorbidity Risk Prediction](https://mlsanity.com/paper/2505.05094/) (2025)
- [Voronoi-grid-based Pareto Front Learning and Its Application to Collaborative Federated Learning](https://mlsanity.com/paper/2505.20648/) (2025)
- [Causal Heterogeneous Graph Learning Method for Chronic Obstructive Pulmonary Disease Prediction](https://mlsanity.com/paper/2512.19194/) (2025)
- [A Novel Graph-Sequence Learning Model for Inductive Text Classification](https://mlsanity.com/paper/2512.20097/) (2025)

## Abstract (arXiv)

In this paper, we present Conjoint Attentions (CAs), a class of novel learning-to-attend strategies for graph neural networks (GNNs). Besides considering the layer-wise node features propagated within the GNN, CAs can additionally incorporate various structural interventions, such as node cluster embedding, and higher-order structural correlations that can be learned outside of GNN, when computing attention scores. The node features that are regarded as significant by the conjoint criteria are therefore more likely to be propagated in the GNN. Given the novel Conjoint Attention strategies, we then propose Graph conjoint attention networks (CATs) that can learn representations embedded with significant latent features deemed by the Conjoint Attentions. Besides, we theoretically validate the discriminative capacity of CATs. CATs utilizing the proposed Conjoint Attention strategies have been extensively tested in well-established benchmarking datasets and comprehensively compared with state-of-the-art baselines. The obtained notable performance demonstrates the effectiveness of the proposed Conjoint Attentions.