# Towards Explaining Hypercomplex Neural Networks

arXiv: https://arxiv.org/abs/2403.17929 · HTML: https://mlsanity.com/paper/2403.17929/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 47 references that are in the map.
- First posted on arXiv 26 Mar 2024.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Maxout Networks](https://mlsanity.com/paper/1302.4389/) (2013)
- [fastai: A Layered API for Deep Learning](https://mlsanity.com/paper/2002.04688/) (2020)
- [Interpretable Convolutional Neural Networks](https://mlsanity.com/paper/1710.00935/) (2017)
- [From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagation](https://mlsanity.com/paper/2206.03208/) (2022)
- [B-cos Networks: Alignment is All We Need for Interpretability](https://mlsanity.com/paper/2205.10268/) (2022)
- [Beyond Fully-Connected Layers with Quaternions: Parameterization of Hypercomplex Multiplications with 1/n Parameters](https://mlsanity.com/paper/2102.08597/) (2021)
- [Convolutional Dynamic Alignment Networks for Interpretable Classifications](https://mlsanity.com/paper/2104.00032/) (2021)
- [Interpretable Convolutional Neural Networks via Feedforward Design](https://mlsanity.com/paper/1810.02786/) (2018)
- [Robust Explainability: A Tutorial on Gradient-Based Attribution Methods for Deep Neural Networks](https://mlsanity.com/paper/2107.11400/) (2021)
- [PHNNs: Lightweight Neural Networks via Parameterized Hypercomplex Convolutions](https://mlsanity.com/paper/2110.04176/) (2021)
- [Universal Approximation Theorem for Vector- and Hypercomplex-Valued Neural Networks](https://mlsanity.com/paper/2401.02277/) (2024)
- [Quaternion-valued Correlation Learning for Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2305.07283/) (2023)
- [Convolutional Filtering and Neural Networks with Non Commutative Algebras](https://mlsanity.com/paper/2108.09923/) (2021)
- [Dual Quaternion Ambisonics Array for Six-Degree-of-Freedom Acoustic Representation](https://mlsanity.com/paper/2204.01851/) (2022)
- [Hypercomplex Multimodal Emotion Recognition from EEG and Peripheral Physiological Signals](https://mlsanity.com/paper/2310.07648/) (2023)
- [Multi-View Hypercomplex Learning for Breast Cancer Screening](https://mlsanity.com/paper/2204.05798/) (2022)
- [Hypercomplex-Valued Recurrent Correlation Neural Networks](https://mlsanity.com/paper/2002.00027/) (2020)
- [Attention-Map Augmentation for Hypercomplex Breast Cancer Classification](https://mlsanity.com/paper/2310.07633/) (2023)
- [Generalizing Medical Image Representations via Quaternion Wavelet Networks](https://mlsanity.com/paper/2310.10224/) (2023)
- [Dual Quaternion Rotational and Translational Equivariance in 3D Rigid Motion Modelling](https://mlsanity.com/paper/2310.07623/) (2023)
- [PHYDI: Initializing Parameterized Hypercomplex Neural Networks as Identity Functions](https://mlsanity.com/paper/2310.07612/) (2023)
- [The HR-Calculus: Enabling Information Processing with Quaternion Algebra](https://mlsanity.com/paper/2311.16771/) (2023)

## Built on by (most cited first, 3 of 3)

- [CVKAN: Complex-Valued Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2502.02417/) (2025)
- [Hierarchical Hypercomplex Network for Multimodal Emotion Recognition](https://mlsanity.com/paper/2409.09194/) (2024)
- [PHemoNet: A Multimodal Network for Physiological Signals](https://mlsanity.com/paper/2410.00010/) (2024)

## Abstract (arXiv)

Hypercomplex neural networks are gaining increasing interest in the deep learning community. The attention directed towards hypercomplex models originates from several aspects, spanning from purely theoretical and mathematical characteristics to the practical advantage of lightweight models over conventional networks, and their unique properties to capture both global and local relations. In particular, a branch of these architectures, parameterized hypercomplex neural networks (PHNNs), has also gained popularity due to their versatility across a multitude of application domains. Nonetheless, only few attempts have been made to explain or interpret their intricacies. In this paper, we propose inherently interpretable PHNNs and quaternion-like networks, thus without the need for any post-hoc method. To achieve this, we define a type of cosine-similarity transform within the parameterized hypercomplex domain. This PHB-cos transform induces weight alignment with relevant input features and allows to reduce the model into a single linear transform, rendering it directly interpretable. In this work, we start to draw insights into how this unique branch of neural models operates. We observe that hypercomplex networks exhibit a tendency to concentrate on the shape around the main object of interest, in addition to the shape of the object itself. We provide a thorough analysis, studying single neurons of different layers and comparing them against how real-valued networks learn. The code of the paper is available at https://github.com/ispamm/HxAI.