# Logic and the 2-Simplicial Transformer

arXiv: https://arxiv.org/abs/1909.00668 · HTML: https://mlsanity.com/paper/1909.00668/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Sep 2019.

## Built on by (most cited first, 9 of 9)

- [Simplicial Hopfield networks](https://mlsanity.com/paper/2305.05179/) (2023)
- [Attending to Topological Spaces: The Cellular Transformer](https://mlsanity.com/paper/2405.14094/) (2024)
- [Fast and Simplex: 2-Simplicial Attention in Triton](https://mlsanity.com/paper/2507.02754/) (2025)
- [Stagewise Reinforcement Learning and the Geometry of the Regret Landscape](https://mlsanity.com/paper/2601.07524/) (2026)
- [Interleaved Head Attention](https://mlsanity.com/paper/2602.21371/) (2026)
- [How Smoothing is N-simplicial Attention?](https://mlsanity.com/paper/2512.15600/) (2025)
- [Beyond Pairwise Attention: Higher-Order Modular Attention for Efficient Sequence Learning](https://mlsanity.com/paper/2603.11133/) (2026)
- [Higher-Order Fourier Neural Operator: Explicit Mode Mixer for Nonlinear PDEs](https://mlsanity.com/paper/2606.28122/) (2026)
- [Linearized 2-Simplicial Attention](https://mlsanity.com/paper/2608.09307/) (2026)

## Abstract (arXiv)

We introduce the 2-simplicial Transformer, an extension of the Transformer which includes a form of higher-dimensional attention generalising the dot-product attention, and uses this attention to update entity representations with tensor products of value vectors. We show that this architecture is a useful inductive bias for logical reasoning in the context of deep reinforcement learning.