# Simplicial Attention Networks

arXiv: https://arxiv.org/abs/2204.09455 · HTML: https://mlsanity.com/paper/2204.09455/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 54 references that are in the map.
- First posted on arXiv 20 Apr 2022.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Link Prediction Based on Graph Neural Networks](https://mlsanity.com/paper/1802.09691/) (2018)
- [Convolutional Networks on Graphs for Learning Molecular Fingerprints](https://mlsanity.com/paper/1509.09292/) (2015)
- [Geometric deep learning on graphs and manifolds using mixture model CNNs](https://mlsanity.com/paper/1611.08402/) (2016)
- [Multi-Label Image Recognition with Graph Convolutional Networks](https://mlsanity.com/paper/1904.03582/) (2019)
- [A Structured Self-attentive Sentence Embedding](https://mlsanity.com/paper/1703.03130/) (2017)
- [Diffusion-Convolutional Neural Networks](https://mlsanity.com/paper/1511.02136/) (2015)
- [A Note on Over-Smoothing for Graph Neural Networks](https://mlsanity.com/paper/2006.13318/) (2020)
- [Weisfeiler and Lehman Go Topological: Message Passing Simplicial Networks](https://mlsanity.com/paper/2103.03212/) (2021)
- [Weisfeiler and Lehman Go Cellular: CW Networks](https://mlsanity.com/paper/2106.12575/) (2021)
- [Long Short-Term Memory-Networks for Machine Reading](https://mlsanity.com/paper/1601.06733/) (2016)
- [Simplicial Neural Networks](https://mlsanity.com/paper/2010.03633/) (2020)
- [Topological Signal Processing over Simplicial Complexes](https://mlsanity.com/paper/1907.11577/) (2019)
- [Random Walks on Simplicial Complexes and the normalized Hodge 1-Laplacian](https://mlsanity.com/paper/1807.05044/) (2018)
- [Signal Processing on Higher-Order Networks: Livin' on the Edge ... and Beyond](https://mlsanity.com/paper/2101.05510/) (2021)
- [Principled Simplicial Neural Networks for Trajectory Prediction](https://mlsanity.com/paper/2102.10058/) (2021)
- [Cell Complex Neural Networks](https://mlsanity.com/paper/2010.00743/) (2020)
- [A Convolutional Encoder Model for Neural Machine Translation](https://mlsanity.com/paper/1611.02344/) (2016)
- [Missing Data Imputation with Adversarially-trained Graph Convolutional Networks](https://mlsanity.com/paper/1905.01907/) (2019)
- [Simplicial Convolutional Neural Networks](https://mlsanity.com/paper/2110.02585/) (2021)
- [HodgeNet: Graph Neural Networks for Edge Data](https://mlsanity.com/paper/1912.02354/) (2019)

## Built on by (most cited first, 30 of 41)

- [Position: Topological Deep Learning is the New Frontier for Relational Learning](https://mlsanity.com/paper/2402.08871/) (2024)
- [Architectures of Topological Deep Learning: A Survey of Message-Passing Topological Neural Networks](https://mlsanity.com/paper/2304.10031/) (2023)
- [Cell Attention Networks](https://mlsanity.com/paper/2209.08179/) (2022)
- [Convolutional Learning on Simplicial Complexes](https://mlsanity.com/paper/2301.11163/) (2023)
- [Generalized Simplicial Attention Neural Networks](https://mlsanity.com/paper/2309.02138/) (2023)
- [E(n) Equivariant Topological Neural Networks](https://mlsanity.com/paper/2405.15429/) (2024)
- [From Latent Graph to Latent Topology Inference: Differentiable Cell Complex Module](https://mlsanity.com/paper/2305.16174/) (2023)
- [TopoTune : A Framework for Generalized Combinatorial Complex Neural Networks](https://mlsanity.com/paper/2410.06530/) (2024)
- [CIN++: Enhancing Topological Message Passing](https://mlsanity.com/paper/2306.03561/) (2023)
- [Copresheaf Topological Neural Networks: A Generalized Deep Learning Framework](https://mlsanity.com/paper/2505.21251/) (2025)
- [Attending to Topological Spaces: The Cellular Transformer](https://mlsanity.com/paper/2405.14094/) (2024)
- [Dirac signal processing of higher-order topological signals](https://mlsanity.com/paper/2301.10137/) (2023)
- [Cycle Invariant Positional Encoding for Graph Representation Learning](https://mlsanity.com/paper/2311.14333/) (2023)
- [ICML 2023 Topological Deep Learning Challenge : Design and Results](https://mlsanity.com/paper/2309.15188/) (2023)
- [Topo-MLP : A Simplicial Network Without Message Passing](https://mlsanity.com/paper/2312.11862/) (2023)
- [CAT-Walk: Inductive Hypergraph Learning via Set Walks](https://mlsanity.com/paper/2306.11147/) (2023)
- [Pooling Strategies for Simplicial Convolutional Networks](https://mlsanity.com/paper/2210.05490/) (2022)
- [Higher-Order Topological Directionality and Directed Simplicial Neural Networks](https://mlsanity.com/paper/2409.08389/) (2024)
- [Topological Flow Matching](https://mlsanity.com/paper/2606.15897/) (2026)
- [Demystifying Higher-Order Graph Neural Networks](https://mlsanity.com/paper/2406.12841/) (2024)
- [Binarized Simplicial Convolutional Neural Networks](https://mlsanity.com/paper/2405.04098/) (2024)
- [Kan Extension Transformers: A Categorical Unification of Attention, Diffusion, and Predict-Detach Self-Conditioning](https://mlsanity.com/paper/2605.27259/) (2026)
- [Stable and Transferable Hyper-Graph Neural Networks](https://mlsanity.com/paper/2211.06513/) (2022)
- [Continuous Simplicial Neural Networks](https://mlsanity.com/paper/2503.12919/) (2025)
- [Topological Adaptive Least Mean Squares Algorithms over Simplicial Complexes](https://mlsanity.com/paper/2505.23160/) (2025)
- [Demystifying Topological Message-Passing with Relational Structures: A Case Study on Oversquashing in Simplicial Message-Passing](https://mlsanity.com/paper/2506.06582/) (2025)
- [Heat Kernel Goes Topological](https://mlsanity.com/paper/2507.12380/) (2025)
- [On the Sheafification of Higher-Order Message Passing](https://mlsanity.com/paper/2509.23020/) (2025)
- [SC-MAD: Mixtures of Higher-order Networks for Data Augmentation](https://mlsanity.com/paper/2309.07453/) (2023)
- [Topological Neural Networks: Mitigating the Bottlenecks of Graph Neural Networks via Higher-Order Interactions](https://mlsanity.com/paper/2402.06908/) (2024)

## Abstract (arXiv)

Graph representation learning methods have mostly been limited to the modelling of node-wise interactions. Recently, there has been an increased interest in understanding how higher-order structures can be utilised to further enhance the learning abilities of graph neural networks (GNNs) in combinatorial spaces. Simplicial Neural Networks (SNNs) naturally model these interactions by performing message passing on simplicial complexes, higher-dimensional generalisations of graphs. Nonetheless, the computations performed by most existent SNNs are strictly tied to the combinatorial structure of the complex. Leveraging the success of attention mechanisms in structured domains, we propose Simplicial Attention Networks (SAT), a new type of simplicial network that dynamically weighs the interactions between neighbouring simplicies and can readily adapt to novel structures. Additionally, we propose a signed attention mechanism that makes SAT orientation equivariant, a desirable property for models operating on (co)chain complexes. We demonstrate that SAT outperforms existent convolutional SNNs and GNNs in two image and trajectory classification tasks.