mlsanityData to 7 Oct 2026

Simplicial Attention Networks

arXiv 2204.09455 ↗First posted 20 Apr 2022cs.LG math.ATcited by 40

Not on the map. Why?

Abstract (arXiv)

Graph representation learning methods have mostly been limited to the modelling of node-wise interactions. Recently, there has been an increased interest in understanding how higher-order structures can be utilised to further enhance the learning abilities of graph neural networks (GNNs) in combinatorial spaces. Simplicial Neural Networks (SNNs) naturally model these interactions by performing message passing on simplicial complexes, higher-dimensional generalisations of graphs. Nonetheless, the computations performed by most existent SNNs are strictly tied to the combinatorial structure of the complex. Leveraging the success of attention mechanisms in structured domains, we propose Simplicial Attention Networks (SAT), a new type of simplicial network that dynamically weighs the interactions between neighbouring simplicies and can readily adapt to novel structures. Additionally, we propose a signed attention mechanism that makes SAT orientation equivariant, a desirable property for models operating on (co)chain complexes. We demonstrate that SAT outperforms existent convolutional SNNs and GNNs in two image and trajectory classification tasks.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Builds on 35 of its 54 references that are in the map. First posted on arXiv 20 Apr 2022.

Builds on

35 of 54 references.

References · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  5. 2017Graph Attention Networkscited by 8,854
  6. 2017Inductive Representation Learning on Large Graphscited by 6,284
  7. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  8. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  9. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  10. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,097
  11. 2018Link Prediction Based on Graph Neural Networkscited by 829
  12. 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 751
23 more
  1. 2016Geometric deep learning on graphs and manifolds using mixture model CNNscited by 468
  2. 2019Multi-Label Image Recognition with Graph Convolutional Networkscited by 320
  3. 2017A Structured Self-attentive Sentence Embeddingcited by 318
  4. 2015Diffusion-Convolutional Neural Networkscited by 268
  5. 2020A Note on Over-Smoothing for Graph Neural Networkscited by 245
  6. 2021Weisfeiler and Lehman Go Topological: Message Passing Simplicial Networkscited by 219
  7. 2021Weisfeiler and Lehman Go Cellular: CW Networkscited by 218
  8. 2016Long Short-Term Memory-Networks for Machine Readingcited by 161
  9. 2020Simplicial Neural Networkscited by 105
  10. 2019Topological Signal Processing over Simplicial Complexescited by 94
  11. 2018Random Walks on Simplicial Complexes and the normalized Hodge 1-Laplaciancited by 84
  12. 2021Signal Processing on Higher-Order Networks: Livin' on the Edge ... and Beyondcited by 81
  13. 2021Principled Simplicial Neural Networks for Trajectory Predictioncited by 77
  14. 2020Cell Complex Neural Networkscited by 66
  15. 2016A Convolutional Encoder Model for Neural Machine Translationcited by 54
  16. 2019Missing Data Imputation with Adversarially-trained Graph Convolutional Networkscited by 54
  17. 2021Simplicial Convolutional Neural Networkscited by 40
  18. 2019HodgeNet: Graph Neural Networks for Edge Datacited by 39
  19. 2021BScNets: Block Simplicial Complex Neural Networkscited by 21
  20. 2022Simplicial Attention Neural Networkscited by 21
  21. 2021Dist2Cycle: A Simplicial Neural Network for Homology Localizationcited by 16
  22. 2021Finite Impulse Response Filters for Simplicial Complexescited by 15
  23. 2019Extrapolating paths with graph neural networkscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 11; 2024: 10; 2025: 9; 2026: 7.

40 papers.

Sort:

By area:

  1. 2024Position: Topological Deep Learning is the New Frontier for Relational Learningcited by 65
  2. 2023Architectures of Topological Deep Learning: A Survey of Message-Passing Topological Neural Networkscited by 43
  3. 2022Cell Attention Networkscited by 34
  4. 2023Generalized Simplicial Attention Neural Networkscited by 27
  5. 2023Convolutional Learning on Simplicial Complexescited by 26
  6. 2024E(n) Equivariant Topological Neural Networkscited by 22
  7. 2023From Latent Graph to Latent Topology Inference: Differentiable Cell Complex Modulecited by 19
  8. 2024TopoTune : A Framework for Generalized Combinatorial Complex Neural Networkscited by 18
  9. 2023CIN++: Enhancing Topological Message Passingcited by 14
  10. 2024Attending to Topological Spaces: The Cellular Transformercited by 11
  11. 2025Copresheaf Topological Neural Networks: A Generalized Deep Learning Frameworkcited by 11
  12. 2023Dirac signal processing of higher-order topological signalscited by 9
28 more
  1. 2023Cycle Invariant Positional Encoding for Graph Representation Learningcited by 8
  2. 2023ICML 2023 Topological Deep Learning Challenge : Design and Resultscited by 7
  3. 2023Topo-MLP : A Simplicial Network Without Message Passingcited by 7
  4. 2023CAT-Walk: Inductive Hypergraph Learning via Set Walkscited by 6
  5. 2022Pooling Strategies for Simplicial Convolutional Networkscited by 5
  6. 2024Higher-Order Topological Directionality and Directed Simplicial Neural Networkscited by 5
  7. 2026Topological Flow Matchingcited by 5
  8. 2024Demystifying Higher-Order Graph Neural Networkscited by 4
  9. 2024Binarized Simplicial Convolutional Neural Networkscited by 3
  10. 2026Kan Extension Transformers: A Categorical Unification of Attention, Diffusion, and Predict-Detach Self-Conditioningcited by 3
  11. 2022Stable and Transferable Hyper-Graph Neural Networkscited by 2
  12. 2025Continuous Simplicial Neural Networkscited by 2
  13. 2025Topological Adaptive Least Mean Squares Algorithms over Simplicial Complexescited by 2
  14. 2025Demystifying Topological Message-Passing with Relational Structures: A Case Study on Oversquashing in Simplicial Message-Passingcited by 2
  15. 2025Heat Kernel Goes Topologicalcited by 1
  16. 2025On the Sheafification of Higher-Order Message Passingcited by 1
  17. 2023SC-MAD: Mixtures of Higher-order Networks for Data Augmentationcited by 0
  18. 2024Topological Neural Networks: Mitigating the Bottlenecks of Graph Neural Networks via Higher-Order Interactionscited by 0
  19. 2024Learning From Simplicial Data Based on Random Walks and 1D Convolutionscited by 0
  20. 2024Higher-order Cross-structural Embedding Model for Time Series Analysiscited by 0
  21. 2025Quotient Complex Transformer (QCformer) for Perovskite Data Analysiscited by 0
  22. 2025Directed Semi-Simplicial Learning with Applications to Brain Activity Decodingcited by 0
  23. 2025Topological Feature Compression for Molecular Graph Neural Networkscited by 0
  24. 2026Topology of Reasoning: Retrieved Cell Complex-Augmented Generation for Textual Graph Question Answeringcited by 0
  25. 2026No Triangulation Without Representation: Generalization in Topological Deep Learningcited by 0
  26. 2026Periodic Topological Deep Learning for Polymer Design and Discoverycited by 0
  27. 2026Collapsed Effective Operators for Higher-order Structurescited by 0
  28. 2026Differentiable Lifting for Topological Neural Networkscited by 0