# Fully tensorial approach to hypercomplex-valued neural networks

arXiv: https://arxiv.org/abs/2407.00449 · HTML: https://mlsanity.com/paper/2407.00449/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 42 references that are in the map.
- First posted on arXiv 29 Jun 2024.

## Builds on (most cited first, 17 of 17)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Quaternion Knowledge Graph Embeddings](https://mlsanity.com/paper/1904.10281/) (2019)
- [TensorLy: Tensor Learning in Python](https://mlsanity.com/paper/1610.09555/) (2016)
- [Fast Algorithms for Convolutional Neural Networks](https://mlsanity.com/paper/1509.09308/) (2015)
- [Deep Quaternion Networks](https://mlsanity.com/paper/1712.04604/) (2017)
- [Quaternion Convolutional Neural Networks](https://mlsanity.com/paper/1903.00658/) (2019)
- [Geometric Clifford Algebra Networks](https://mlsanity.com/paper/2302.06594/) (2023)
- [PHNNs: Lightweight Neural Networks via Parameterized Hypercomplex Convolutions](https://mlsanity.com/paper/2110.04176/) (2021)
- [Algebraic Neural Networks: Stability to Deformations](https://mlsanity.com/paper/2009.01433/) (2020)
- [Quaternion Convolutional Neural Networks for End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/1806.07789/) (2018)
- [Universal Approximation Theorem for Vector- and Hypercomplex-Valued Neural Networks](https://mlsanity.com/paper/2401.02277/) (2024)
- [Quaternion Collaborative Filtering for Recommendation](https://mlsanity.com/paper/1906.02594/) (2019)
- [Knowledge Graph Embeddings in Geometric Algebras](https://mlsanity.com/paper/2010.00989/) (2020)
- [Multi-View Hypercomplex Learning for Breast Cancer Screening](https://mlsanity.com/paper/2204.05798/) (2022)
- [Hypercomplex-Valued Recurrent Correlation Neural Networks](https://mlsanity.com/paper/2002.00027/) (2020)
- [Quaternion Matrix Completion Using Untrained Quaternion Convolutional Neural Network for Color Image Inpainting](https://mlsanity.com/paper/2305.00416/) (2023)
- [Algebraic foundations of split hypercomplex nonlinear adaptive filtering](https://mlsanity.com/paper/1306.1676/) (2013)

## Built on by (most cited first, 1 of 1)

- [Change the Product, Keep the Parameters: Associative Algebra Layers for Transformers](https://mlsanity.com/paper/2609.32814/) (2026)

## Abstract (arXiv)

A fully tensorial theoretical framework for hypercomplex-valued neural networks is presented. The proposed approach enables neural network architectures to operate on data defined over arbitrary finite-dimensional algebras. The central observation is that algebra multiplication can be represented by a rank-three tensor, which allows all algebraic operations in neural network layers to be formulated in terms of standard tensor contractions, permutations, and reshaping operations. This tensor-based formulation provides a unified and dimension-independent description of hypercomplex-valued dense and convolutional layers and is directly compatible with modern deep learning libraries supporting optimized tensor operations. The proposed framework recovers existing constructions for four-dimensional algebras as a special case. Within this setting, a tensor-based version of the universal approximation theorem for single-layer hypercomplex-valued perceptrons is established under mild non-degeneracy assumptions on the underlying algebra, thereby providing a rigorous theoretical foundation for the considered class of neural networks.