# Simplicial Attention Neural Networks

arXiv: https://arxiv.org/abs/2203.07485 · HTML: https://mlsanity.com/paper/2203.07485/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 39 references that are in the map.
- First posted on arXiv 14 Mar 2022.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Link Prediction Based on Graph Neural Networks](https://mlsanity.com/paper/1802.09691/) (2018)
- [Convolutional Networks on Graphs for Learning Molecular Fingerprints](https://mlsanity.com/paper/1509.09292/) (2015)
- [Graph Transformer Networks](https://mlsanity.com/paper/1911.06455/) (2019)
- [Multi-Label Image Recognition with Graph Convolutional Networks](https://mlsanity.com/paper/1904.03582/) (2019)
- [A Structured Self-attentive Sentence Embedding](https://mlsanity.com/paper/1703.03130/) (2017)
- [Diffusion-Convolutional Neural Networks](https://mlsanity.com/paper/1511.02136/) (2015)
- [Weisfeiler and Lehman Go Topological: Message Passing Simplicial Networks](https://mlsanity.com/paper/2103.03212/) (2021)
- [Long Short-Term Memory-Networks for Machine Reading](https://mlsanity.com/paper/1601.06733/) (2016)
- [Simplicial Neural Networks](https://mlsanity.com/paper/2010.03633/) (2020)
- [Topological Signal Processing over Simplicial Complexes](https://mlsanity.com/paper/1907.11577/) (2019)
- [Random Walks on Simplicial Complexes and the normalized Hodge 1-Laplacian](https://mlsanity.com/paper/1807.05044/) (2018)
- [Signal Processing on Higher-Order Networks: Livin' on the Edge ... and Beyond](https://mlsanity.com/paper/2101.05510/) (2021)
- [Principled Simplicial Neural Networks for Trajectory Prediction](https://mlsanity.com/paper/2102.10058/) (2021)
- [A Convolutional Encoder Model for Neural Machine Translation](https://mlsanity.com/paper/1611.02344/) (2016)
- [Missing Data Imputation with Adversarially-trained Graph Convolutional Networks](https://mlsanity.com/paper/1905.01907/) (2019)
- [Simplicial Convolutional Neural Networks](https://mlsanity.com/paper/2110.02585/) (2021)
- [HodgeNet: Graph Neural Networks for Edge Data](https://mlsanity.com/paper/1912.02354/) (2019)
- [Finite Impulse Response Filters for Simplicial Complexes](https://mlsanity.com/paper/2103.12587/) (2021)

## Built on by (most cited first, 21 of 21)

- [Position: Topological Deep Learning is the New Frontier for Relational Learning](https://mlsanity.com/paper/2402.08871/) (2024)
- [Simplicial Attention Networks](https://mlsanity.com/paper/2204.09455/) (2022)
- [Cell Attention Networks](https://mlsanity.com/paper/2209.08179/) (2022)
- [Generalized Simplicial Attention Neural Networks](https://mlsanity.com/paper/2309.02138/) (2023)
- [TopoBench: A Framework for Benchmarking Topological Deep Learning](https://mlsanity.com/paper/2406.06642/) (2024)
- [Tangent Bundle Convolutional Learning: from Manifolds to Cellular Sheaves and Back](https://mlsanity.com/paper/2303.11323/) (2023)
- [E(n) Equivariant Topological Neural Networks](https://mlsanity.com/paper/2405.15429/) (2024)
- [From Latent Graph to Latent Topology Inference: Differentiable Cell Complex Module](https://mlsanity.com/paper/2305.16174/) (2023)
- [Tangent Bundle Filters and Neural Networks: from Manifolds to Cellular Sheaves and Back](https://mlsanity.com/paper/2210.15058/) (2022)
- [On the Theoretical Expressive Power and the Design Space of Higher-Order Graph Transformers](https://mlsanity.com/paper/2404.03380/) (2024)
- [Topo-MLP : A Simplicial Network Without Message Passing](https://mlsanity.com/paper/2312.11862/) (2023)
- [Pooling Strategies for Simplicial Convolutional Networks](https://mlsanity.com/paper/2210.05490/) (2022)
- [Representation Power of Graph Neural Networks: Improved Expressivity via Algebraic Analysis](https://mlsanity.com/paper/2205.09801/) (2022)
- [Binarized Simplicial Convolutional Neural Networks](https://mlsanity.com/paper/2405.04098/) (2024)
- [Graph Neural Networks for Edge Signals: Orientation Equivariance and Invariance](https://mlsanity.com/paper/2410.16935/) (2024)
- [Continuous Simplicial Neural Networks](https://mlsanity.com/paper/2503.12919/) (2025)
- [Topological Neural Networks: Mitigating the Bottlenecks of Graph Neural Networks via Higher-Order Interactions](https://mlsanity.com/paper/2402.06908/) (2024)
- [Learning From Simplicial Data Based on Random Walks and 1D Convolutions](https://mlsanity.com/paper/2404.03434/) (2024)
- [Topological Neural Networks over the Air](https://mlsanity.com/paper/2502.10070/) (2025)
- [Directed Semi-Simplicial Learning with Applications to Brain Activity Decoding](https://mlsanity.com/paper/2505.17939/) (2025)
- [TopoU-Net: a U-Net architecture for topological domains](https://mlsanity.com/paper/2605.10091/) (2026)

## Abstract (arXiv)

The aim of this work is to introduce simplicial attention networks (SANs), i.e., novel neural architectures that operate on data defined on simplicial complexes leveraging masked self-attentional layers. Hinging on formal arguments from topological signal processing, we introduce a proper self-attention mechanism able to process data components at different layers (e.g., nodes, edges, triangles, and so on), while learning how to weight both upper and lower neighborhoods of the given topological domain in a totally task-oriented fashion. The proposed SANs generalize most of the current architectures available for processing data defined on simplicial complexes. The proposed approach compares favorably with other methods when applied to different (inductive and transductive) tasks such as trajectory prediction and missing data imputations in citation complexes.