# Generalized Simplicial Attention Neural Networks

arXiv: https://arxiv.org/abs/2309.02138 · HTML: https://mlsanity.com/paper/2309.02138/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 69 references that are in the map.
- First posted on arXiv 5 Sep 2023.

## Builds on (most cited first, 30 of 46)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [How Attentive are Graph Attention Networks?](https://mlsanity.com/paper/2105.14491/) (2021)
- [Link Prediction Based on Graph Neural Networks](https://mlsanity.com/paper/1802.09691/) (2018)
- [Convolutional Networks on Graphs for Learning Molecular Fingerprints](https://mlsanity.com/paper/1509.09292/) (2015)
- [TUDataset: A collection of benchmark datasets for learning with graphs](https://mlsanity.com/paper/2007.08663/) (2020)
- [Graph Transformer Networks](https://mlsanity.com/paper/1911.06455/) (2019)
- [Multi-Label Image Recognition with Graph Convolutional Networks](https://mlsanity.com/paper/1904.03582/) (2019)
- [A Structured Self-attentive Sentence Embedding](https://mlsanity.com/paper/1703.03130/) (2017)
- [Weisfeiler and Lehman Go Topological: Message Passing Simplicial Networks](https://mlsanity.com/paper/2103.03212/) (2021)
- [Long Short-Term Memory-Networks for Machine Reading](https://mlsanity.com/paper/1601.06733/) (2016)
- [Stability Properties of Graph Neural Networks](https://mlsanity.com/paper/1905.04497/) (2019)
- [Simplicial Neural Networks](https://mlsanity.com/paper/2010.03633/) (2020)
- [Topological Deep Learning: Going Beyond Graph Data](https://mlsanity.com/paper/2206.00606/) (2022)
- [Topological Signal Processing over Simplicial Complexes](https://mlsanity.com/paper/1907.11577/) (2019)
- [Learning shape correspondence with anisotropic convolutional neural networks](https://mlsanity.com/paper/1605.06437/) (2016)
- [Convolutional Neural Network Architectures for Signals Supported on Graphs](https://mlsanity.com/paper/1805.00165/) (2018)
- [Random Walks on Simplicial Complexes and the normalized Hodge 1-Laplacian](https://mlsanity.com/paper/1807.05044/) (2018)
- [Signal Processing on Higher-Order Networks: Livin' on the Edge ... and Beyond](https://mlsanity.com/paper/2101.05510/) (2021)
- [Principled Simplicial Neural Networks for Trajectory Prediction](https://mlsanity.com/paper/2102.10058/) (2021)
- [A Convolutional Encoder Model for Neural Machine Translation](https://mlsanity.com/paper/1611.02344/) (2016)
- [Missing Data Imputation with Adversarially-trained Graph Convolutional Networks](https://mlsanity.com/paper/1905.01907/) (2019)
- [Architectures of Topological Deep Learning: A Survey of Message-Passing Topological Neural Networks](https://mlsanity.com/paper/2304.10031/) (2023)

## Built on by (most cited first, 27 of 27)

- [Position: Topological Deep Learning is the New Frontier for Relational Learning](https://mlsanity.com/paper/2402.08871/) (2024)
- [TopoBench: A Framework for Benchmarking Topological Deep Learning](https://mlsanity.com/paper/2406.06642/) (2024)
- [E(n) Equivariant Topological Neural Networks](https://mlsanity.com/paper/2405.15429/) (2024)
- [TopoTune : A Framework for Generalized Combinatorial Complex Neural Networks](https://mlsanity.com/paper/2410.06530/) (2024)
- [Gaussian Processes on Cellular Complexes](https://mlsanity.com/paper/2311.01198/) (2023)
- [Copresheaf Topological Neural Networks: A Generalized Deep Learning Framework](https://mlsanity.com/paper/2505.21251/) (2025)
- [Hypergraph-MLP: Learning on Hypergraphs without Message Passing](https://mlsanity.com/paper/2312.09778/) (2023)
- [Higher-Order Topological Directionality and Directed Simplicial Neural Networks](https://mlsanity.com/paper/2409.08389/) (2024)
- [Topological Flow Matching](https://mlsanity.com/paper/2606.15897/) (2026)
- [Graph Classification Gaussian Processes via Hodgelet Spectral Features](https://mlsanity.com/paper/2410.10546/) (2024)
- [Matched Topological Subspace Detector](https://mlsanity.com/paper/2504.05892/) (2025)
- [Kan Extension Transformers: A Categorical Unification of Attention, Diffusion, and Predict-Detach Self-Conditioning](https://mlsanity.com/paper/2605.27259/) (2026)
- [Continuous Simplicial Neural Networks](https://mlsanity.com/paper/2503.12919/) (2025)
- [Topological Adaptive Least Mean Squares Algorithms over Simplicial Complexes](https://mlsanity.com/paper/2505.23160/) (2025)
- [Simplicial Gaussian Models: Representation and Inference](https://mlsanity.com/paper/2510.12983/) (2025)
- [Advancing Graph Neural Networks with HL-HGAT: A Hodge-Laplacian and Attention Mechanism Approach for Heterogeneous Graph-Structured Data](https://mlsanity.com/paper/2403.06687/) (2024)
- [Dirac-Equation Signal Processing: Physics Boosts Topological Machine Learning](https://mlsanity.com/paper/2412.05132/) (2024)
- [Heat Kernel Goes Topological](https://mlsanity.com/paper/2507.12380/) (2025)
- [On the Sheafification of Higher-Order Message Passing](https://mlsanity.com/paper/2509.23020/) (2025)
- [Stationarity and Spectral Characterization of Random Signals on Simplicial Complexes](https://mlsanity.com/paper/2602.03055/) (2026)
- [Directed Semi-Simplicial Learning with Applications to Brain Activity Decoding](https://mlsanity.com/paper/2505.17939/) (2025)
- [TopoU-Net: a U-Net architecture for topological domains](https://mlsanity.com/paper/2605.10091/) (2026)
- [Topological Kalman Filtering on Cell Complexes](https://mlsanity.com/paper/2605.15955/) (2026)
- [Topological Signal Processing: An Application-Oriented Tutorial](https://mlsanity.com/paper/2605.22853/) (2026)
- [Scaling Higher-Order Graph Learning with Maximal Clique Complexes](https://mlsanity.com/paper/2605.31373/) (2026)
- [Sheaf-theoretic Signal Processing on Graphs: Spectral Theory, Filtering, and Sampling](https://mlsanity.com/paper/2608.01318/) (2026)
- [Demystifying Oversmoothing in Sheaf Neural Networks: An Index-Theoretic Criterion](https://mlsanity.com/paper/2608.16180/) (2026)

## Abstract (arXiv)

Graph machine learning methods excel at leveraging pairwise relations present in the data. However, graphs are unable to fully capture the multi-way interactions inherent in many complex systems. An effective way to incorporate them is to model the data on higher-order combinatorial topological spaces, such as Simplicial Complexes (SCs) or Cell Complexes. For this reason, we introduce Generalized Simplicial Attention Neural Networks (GSANs), novel neural network architectures designed to process data living on simplicial complexes using masked self-attentional layers. Hinging on topological signal processing principles, we devise a series of principled self-attention mechanisms able to process data associated with simplices of various order, such as nodes, edges, triangles, and beyond. These schemes learn how to combine data associated with neighbor simplices of consecutive order in a task-oriented fashion, leveraging on the simplicial Dirac operator and its Dirac decomposition. We also prove that GSAN satisfies two fundamental properties: permutation equivariance and simplicial-awareness. Finally, we illustrate how our approach compares favorably with other simplicial and graph models when applied to several (inductive and transductive) tasks such as trajectory prediction, missing data imputation, graph classification, and simplex prediction.