# PHNNs: Lightweight Neural Networks via Parameterized Hypercomplex Convolutions

arXiv: https://arxiv.org/abs/2110.04176 · HTML: https://mlsanity.com/paper/2110.04176/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 61 references that are in the map.
- First posted on arXiv 8 Oct 2021.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases](https://mlsanity.com/paper/2103.10697/) (2021)
- [Compacter: Efficient Low-Rank Hypercomplex Adapter Layers](https://mlsanity.com/paper/2106.04647/) (2021)
- [ResNet strikes back: An improved training procedure in timm](https://mlsanity.com/paper/2110.00476/) (2021)
- [Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks](https://mlsanity.com/paper/1807.00129/) (2018)
- [Beyond Fully-Connected Layers with Quaternions: Parameterization of Hypercomplex Multiplications with 1/n Parameters](https://mlsanity.com/paper/2102.08597/) (2021)
- [Sound Event Detection: A Tutorial](https://mlsanity.com/paper/2107.05463/) (2021)
- [Deep Quaternion Networks](https://mlsanity.com/paper/1712.04604/) (2017)
- [Quaternion Recurrent Neural Networks](https://mlsanity.com/paper/1806.04418/) (2018)
- [Quaternion Convolutional Neural Networks for Heterogeneous Image Processing](https://mlsanity.com/paper/1811.02656/) (2018)
- [Lightweight and Efficient Neural Natural Language Processing with Quaternion Networks](https://mlsanity.com/paper/1906.04393/) (2019)
- [Quaternion Convolutional Neural Networks for Detection and Localization of 3D Sound Events](https://mlsanity.com/paper/1812.06811/) (2018)
- [Quaternion Generative Adversarial Networks](https://mlsanity.com/paper/2104.09630/) (2021)
- [Kronecker CP Decomposition with Fast Multiplication for Compressing RNNs](https://mlsanity.com/paper/2008.09342/) (2020)
- [A Quaternion-Valued Variational Autoencoder](https://mlsanity.com/paper/2010.11647/) (2020)
- [L3DAS21 Challenge: Machine Learning for 3D Audio Signal Processing](https://mlsanity.com/paper/2104.05499/) (2021)
- [Deep Octonion Networks](https://mlsanity.com/paper/1903.08478/) (2019)
- [AlgebraNets](https://mlsanity.com/paper/2006.07360/) (2020)
- [Hypercomplex-Valued Recurrent Correlation Neural Networks](https://mlsanity.com/paper/2002.00027/) (2020)
- [A Broad Class of Discrete-Time Hypercomplex-Valued Hopfield Neural Networks](https://mlsanity.com/paper/1902.05478/) (2019)
- [Quaternion Factorization Machines: A Lightweight Solution to Intricate Feature Interaction Modelling](https://mlsanity.com/paper/2104.01716/) (2021)
- [Stochastic Neural Network with Kronecker Flow](https://mlsanity.com/paper/1906.04282/) (2019)

## Built on by (most cited first, 26 of 26)

- [Parameter-Efficient Fine-Tuning for Foundation Models](https://mlsanity.com/paper/2501.13787/) (2025)
- [Visual Tuning](https://mlsanity.com/paper/2305.06061/) (2023)
- [Universal Approximation Theorem for Vector- and Hypercomplex-Valued Neural Networks](https://mlsanity.com/paper/2401.02277/) (2024)
- [Dual Quaternion Ambisonics Array for Six-Degree-of-Freedom Acoustic Representation](https://mlsanity.com/paper/2204.01851/) (2022)
- [Hypercomplex Multimodal Emotion Recognition from EEG and Peripheral Physiological Signals](https://mlsanity.com/paper/2310.07648/) (2023)
- [Learning Speech Emotion Representations in the Quaternion Domain](https://mlsanity.com/paper/2204.02385/) (2022)
- [Multi-View Hypercomplex Learning for Breast Cancer Screening](https://mlsanity.com/paper/2204.05798/) (2022)
- [Attention-Map Augmentation for Hypercomplex Breast Cancer Classification](https://mlsanity.com/paper/2310.07633/) (2023)
- [Hypercomplex Image-to-Image Translation](https://mlsanity.com/paper/2205.02087/) (2022)
- [Understanding Vector-Valued Neural Networks and Their Relationship with Real and Hypercomplex-Valued Neural Networks](https://mlsanity.com/paper/2309.07716/) (2023)
- [Hierarchical Hypercomplex Network for Multimodal Emotion Recognition](https://mlsanity.com/paper/2409.09194/) (2024)
- [Quaternion Convolutional Neural Networks: Current Advances and Future Directions](https://mlsanity.com/paper/2307.08663/) (2023)
- [Towards Explaining Hypercomplex Neural Networks](https://mlsanity.com/paper/2403.17929/) (2024)
- [PHYDI: Initializing Parameterized Hypercomplex Neural Networks as Identity Functions](https://mlsanity.com/paper/2310.07612/) (2023)
- [PHemoNet: A Multimodal Network for Physiological Signals](https://mlsanity.com/paper/2410.00010/) (2024)
- [Efficient ECG-based Atrial Fibrillation Detection via Parameterised Hypercomplex Neural Networks](https://mlsanity.com/paper/2211.02678/) (2022)
- [Enhancing Semantic Communication with Deep Generative Models -- An ICASSP Special Session Overview](https://mlsanity.com/paper/2309.02478/) (2023)
- [Demystifying the Hypercomplex: Inductive Biases in Hypercomplex Deep Learning](https://mlsanity.com/paper/2405.07024/) (2024)
- [Improving Quaternion Neural Networks with Quaternionic Activation Functions](https://mlsanity.com/paper/2406.16481/) (2024)
- [Fully tensorial approach to hypercomplex-valued neural networks](https://mlsanity.com/paper/2407.00449/) (2024)
- [ToTMNet: FFT-Accelerated Toeplitz Temporal Mixing Network for Lightweight Remote Photoplethysmography](https://mlsanity.com/paper/2601.04159/) (2026)
- [KHNNs: hypercomplex neural networks computations via Keras using TensorFlow and PyTorch](https://mlsanity.com/paper/2407.00452/) (2024)
- [V-EfficientNets: Vector-Valued Efficiently Scaled Convolutional Neural Network Models](https://mlsanity.com/paper/2505.05659/) (2025)
- [Compressing Quaternion Convolutional Neural Networks for Audio Classification](https://mlsanity.com/paper/2510.21388/) (2025)
- [EdgeLDR: Quaternion Low-Displacement Rank Neural Networks for Edge-Efficient Deep Learning](https://mlsanity.com/paper/2601.05379/) (2026)
- [Ghost Features and Spooky Transfer Learning for Hypercomplex-Valued Neural Networks](https://mlsanity.com/paper/2608.07735/) (2026)

## Abstract (arXiv)

Hypercomplex neural networks have proven to reduce the overall number of parameters while ensuring valuable performance by leveraging the properties of Clifford algebras. Recently, hypercomplex linear layers have been further improved by involving efficient parameterized Kronecker products. In this paper, we define the parameterization of hypercomplex convolutional layers and introduce the family of parameterized hypercomplex neural networks (PHNNs) that are lightweight and efficient large-scale models. Our method grasps the convolution rules and the filter organization directly from data without requiring a rigidly predefined domain structure to follow. PHNNs are flexible to operate in any user-defined or tuned domain, from 1D to nD regardless of whether the algebra rules are preset. Such a malleability allows processing multidimensional inputs in their natural domain without annexing further dimensions, as done, instead, in quaternion neural networks for 3D inputs like color images. As a result, the proposed family of PHNNs operates with 1/n free parameters as regards its analog in the real domain. We demonstrate the versatility of this approach to multiple domains of application by performing experiments on various image datasets as well as audio datasets in which our method outperforms real and quaternion-valued counterparts. Full code is available at: https://github.com/eleGAN23/HyperNets.