# Demystifying the Hypercomplex: Inductive Biases in Hypercomplex Deep Learning

arXiv: https://arxiv.org/abs/2405.07024 · HTML: https://mlsanity.com/paper/2405.07024/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 31 references that are in the map.
- First posted on arXiv 11 May 2024.

## Builds on (most cited first, 19 of 19)

- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Attention Bottlenecks for Multimodal Fusion](https://mlsanity.com/paper/2107.00135/) (2021)
- [Inductive Biases for Deep Learning of Higher-Level Cognition](https://mlsanity.com/paper/2011.15091/) (2020)
- [Deep Multimodal Fusion by Channel Exchanging](https://mlsanity.com/paper/2011.05005/) (2020)
- [Beyond Fully-Connected Layers with Quaternions: Parameterization of Hypercomplex Multiplications with 1/n Parameters](https://mlsanity.com/paper/2102.08597/) (2021)
- [The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learning](https://mlsanity.com/paper/2304.05366/) (2023)
- [Lightweight and Efficient Neural Natural Language Processing with Quaternion Networks](https://mlsanity.com/paper/1906.04393/) (2019)
- [PHNNs: Lightweight Neural Networks via Parameterized Hypercomplex Convolutions](https://mlsanity.com/paper/2110.04176/) (2021)
- [Quaternion Convolutional Neural Networks for Detection and Localization of 3D Sound Events](https://mlsanity.com/paper/1812.06811/) (2018)
- [Quaternion Generative Adversarial Networks](https://mlsanity.com/paper/2104.09630/) (2021)
- [A Quaternion-Valued Variational Autoencoder](https://mlsanity.com/paper/2010.11647/) (2020)
- [Speech Emotion Recognition Using Quaternion Convolutional Neural Networks](https://mlsanity.com/paper/2111.00404/) (2021)
- [Quaternion Graph Neural Networks](https://mlsanity.com/paper/2008.05089/) (2020)
- [Dual Quaternion Ambisonics Array for Six-Degree-of-Freedom Acoustic Representation](https://mlsanity.com/paper/2204.01851/) (2022)
- [Learning Speech Emotion Representations in the Quaternion Domain](https://mlsanity.com/paper/2204.02385/) (2022)
- [Multi-View Hypercomplex Learning for Breast Cancer Screening](https://mlsanity.com/paper/2204.05798/) (2022)
- [Attention-Map Augmentation for Hypercomplex Breast Cancer Classification](https://mlsanity.com/paper/2310.07633/) (2023)
- [How can classical multidimensional scaling go wrong?](https://mlsanity.com/paper/2110.11430/) (2021)
- [Dual Quaternion Rotational and Translational Equivariance in 3D Rigid Motion Modelling](https://mlsanity.com/paper/2310.07623/) (2023)

## Built on by (most cited first, 1 of 1)

- [GeoPE:A Unified Geometric Positional Embedding for Structured Tensors](https://mlsanity.com/paper/2512.04963/) (2025)

## Abstract (arXiv)

Hypercomplex algebras have recently been gaining prominence in the field of deep learning owing to the advantages of their division algebras over real vector spaces and their superior results when dealing with multidimensional signals in real-world 3D and 4D paradigms. This paper provides a foundational framework that serves as a roadmap for understanding why hypercomplex deep learning methods are so successful and how their potential can be exploited. Such a theoretical framework is described in terms of inductive bias, i.e., a collection of assumptions, properties, and constraints that are built into training algorithms to guide their learning process toward more efficient and accurate solutions. We show that it is possible to derive specific inductive biases in the hypercomplex domains, which extend complex numbers to encompass diverse numbers and data structures. These biases prove effective in managing the distinctive properties of these domains, as well as the complex structures of multidimensional and multimodal signals. This novel perspective for hypercomplex deep learning promises to both demystify this class of methods and clarify their potential, under a unifying framework, and in this way promotes hypercomplex models as viable alternatives to traditional real-valued deep learning for multidimensional signal processing.