# Deep Axial Hypercomplex Networks

arXiv: https://arxiv.org/abs/2301.04626 · HTML: https://mlsanity.com/paper/2301.04626/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 22 references that are in the map.
- First posted on arXiv 11 Jan 2023.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Axial Attention in Multidimensional Transformers](https://mlsanity.com/paper/1912.12180/) (2019)
- [Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation](https://mlsanity.com/paper/2003.07853/) (2020)
- [Unitary Evolution Recurrent Neural Networks](https://mlsanity.com/paper/1511.06464/) (2015)
- [Deep Complex Networks](https://mlsanity.com/paper/1705.09792/) (2017)
- [Beyond Fully-Connected Layers with Quaternions: Parameterization of Hypercomplex Multiplications with 1/n Parameters](https://mlsanity.com/paper/2102.08597/) (2021)
- [Deep Quaternion Networks](https://mlsanity.com/paper/1712.04604/) (2017)
- [Quaternion Recurrent Neural Networks](https://mlsanity.com/paper/1806.04418/) (2018)
- [Associative Long Short-Term Memory](https://mlsanity.com/paper/1602.03032/) (2016)
- [Quaternion Convolutional Neural Networks for Heterogeneous Image Processing](https://mlsanity.com/paper/1811.02656/) (2018)
- [Deep Octonion Networks](https://mlsanity.com/paper/1903.08478/) (2019)
- [Complex Network Classification with Convolutional Neural Network](https://mlsanity.com/paper/1802.00539/) (2018)

## Abstract (arXiv)

Over the past decade, deep hypercomplex-inspired networks have enhanced feature extraction for image classification by enabling weight sharing across input channels. Recent works make it possible to improve representational capabilities by using hypercomplex-inspired networks which consume high computational costs. This paper reduces this cost by factorizing a quaternion 2D convolutional module into two consecutive vectormap 1D convolutional modules. Also, we use 5D parameterized hypercomplex multiplication based fully connected layers. Incorporating both yields our proposed hypercomplex network, a novel architecture that can be assembled to construct deep axial-hypercomplex networks (DANs) for image classifications. We conduct experiments on CIFAR benchmarks, SVHN, and Tiny ImageNet datasets and achieve better performance with fewer trainable parameters and FLOPS. Our proposed model achieves almost 2% higher performance for CIFAR and SVHN datasets, and more than 3% for the ImageNet-Tiny dataset and takes six times fewer parameters than the real-valued ResNets. Also, it shows state-of-the-art performance on CIFAR benchmarks in hypercomplex space.