Product Interaction: An Algebraic Formalism for Deep Learning Architectures
Not on the map. Why?
Abstract (arXiv)
In this paper, we introduce product interactions, an algebraic formalism in which neural network layers are constructed from compositions of a multiplication operator defined over suitable algebras. Product interactions provide a principled way to generate and organize algebraic expressions by increasing interaction order. Our central observation is that algebraic expressions in modern neural networks admit a unified construction in terms of linear, quadratic, and higher-order product interactions. Convolutional and equivariant networks arise as symmetry-constrained linear product interactions, while attention and Mamba correspond to higher-order product interactions.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 36 references that are in the map. First posted on arXiv 31 Jan 2026.
Builds on
26 of 36 references.
References · 26
- 2017Attention Is All You Needcited by 59,932
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
- 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
- 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
- 2020Rethinking Attention with Performerscited by 1,584
- 2016Group Equivariant Convolutional Networkscited by 1,154
- 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
- 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
- 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 735
- 2021E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentialscited by 701
- 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 602
14 more
- 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 594
- 2022Hungry Hungry Hippos: Towards Language Modeling with State Space Modelscited by 495
- 20183D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Datacited by 348
- 2016Harmonic Networks: Deep Translation and Rotation Equivariancecited by 293
- 2019Gauge Equivariant Convolutional Networks and the Icosahedral CNNcited by 262
- 2018A General Theory of Equivariant CNNs on Homogeneous Spacescited by 223
- 2019Pay Less Attention with Lightweight and Dynamic Convolutionscited by 211
- 2018Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Networkcited by 168
- 2019Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of Viewcited by 108
- 2020LieTransformer: Equivariant self-attention for Lie Groupscited by 100
- 2024Position: Categorical Deep Learning is an Algebraic Theory of All Architecturescited by 32
- 2025Tensor Product Attention Is All You Needcited by 27
- 2021X-volution: On the unification of convolution and self-attentioncited by 9
- 2025Block-Biased Mamba for Long-Range Sequence Processingcited by 7
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2023Deep Tensor Network
- 2026Low-Interaction-Rank Learning: Unifying Multiplicative Dual-Encoder Heads
- 2026Dynamics Reveals Structure: Challenging the Linear Propagation Assumption
- 2025Why Neural Network Can Discover Symbolic Structures with Gradient-based Training: An Algebraic and Geometric Foundation for Neurosymbolic Reasoning
- 2026Neural Quadratic Forms: A Unified Minimal Model for Sudden Learning and Scaling Laws
- 2026BRo-JEPA: Learning Modular Transformations in Latent Space
- 2026ITNet: A Learnable Integral Transform That Subsumes Convolution, Attention, and Recurrence
- 2026An Algebraic View of the Expressivity of Recurrent Language Models