# Deep Quaternion Networks

arXiv: https://arxiv.org/abs/1712.04604 · HTML: https://mlsanity.com/paper/1712.04604/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Dec 2017.

## Built on by (most cited first, 30 of 48)

- [Compacter: Efficient Low-Rank Hypercomplex Adapter Layers](https://mlsanity.com/paper/2106.04647/) (2021)
- [The Unsurprising Effectiveness of Pre-Trained Vision Models for Control](https://mlsanity.com/paper/2203.03580/) (2022)
- [Clifford Neural Layers for PDE Modeling](https://mlsanity.com/paper/2209.04934/) (2022)
- [Beyond Fully-Connected Layers with Quaternions: Parameterization of Hypercomplex Multiplications with 1/n Parameters](https://mlsanity.com/paper/2102.08597/) (2021)
- [PHNNs: Lightweight Neural Networks via Parameterized Hypercomplex Convolutions](https://mlsanity.com/paper/2110.04176/) (2021)
- [Quaternion Generative Adversarial Networks](https://mlsanity.com/paper/2104.09630/) (2021)
- [Quaternion-valued Correlation Learning for Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2305.07283/) (2023)
- [Convolutional Filtering and Neural Networks with Non Commutative Algebras](https://mlsanity.com/paper/2108.09923/) (2021)
- [Color Image Inpainting via Robust Pure Quaternion Matrix Completion: Error Bound and Weighted Loss](https://mlsanity.com/paper/2202.02063/) (2022)
- [Dual Quaternion Ambisonics Array for Six-Degree-of-Freedom Acoustic Representation](https://mlsanity.com/paper/2204.01851/) (2022)
- [Acute Lymphoblastic Leukemia Detection Using Hypercomplex-Valued Convolutional Neural Networks](https://mlsanity.com/paper/2205.13273/) (2022)
- [Parameterized Hypercomplex Graph Neural Networks for Graph Classification](https://mlsanity.com/paper/2103.16584/) (2021)
- [Learning Speech Emotion Representations in the Quaternion Domain](https://mlsanity.com/paper/2204.02385/) (2022)
- [Multi-View Hypercomplex Learning for Breast Cancer Screening](https://mlsanity.com/paper/2204.05798/) (2022)
- [Color Equivariant Convolutional Networks](https://mlsanity.com/paper/2310.19368/) (2023)
- [Q-RBSA: High-Resolution 3D EBSD Map Generation Using An Efficient Quaternion Transformer Network](https://mlsanity.com/paper/2303.10722/) (2023)
- [Quaternion Factorization Machines: A Lightweight Solution to Intricate Feature Interaction Modelling](https://mlsanity.com/paper/2104.01716/) (2021)
- [Noncommutative C^*-algebra Net: Learning Neural Networks with Powerful Product Structure in C^*-algebra](https://mlsanity.com/paper/2302.01191/) (2023)
- [Understanding Vector-Valued Neural Networks and Their Relationship with Real and Hypercomplex-Valued Neural Networks](https://mlsanity.com/paper/2309.07716/) (2023)
- [RingCNN: Exploiting Algebraically-Sparse Ring Tensors for Energy-Efficient CNN-Based Computational Imaging](https://mlsanity.com/paper/2104.09056/) (2021)
- [Two-view Graph Neural Networks for Knowledge Graph Completion](https://mlsanity.com/paper/2112.09231/) (2021)
- [Quaternion Convolutional Neural Networks: Current Advances and Future Directions](https://mlsanity.com/paper/2307.08663/) (2023)
- [Time Series Compression using Quaternion Valued Neural Networks and Quaternion Backpropagation](https://mlsanity.com/paper/2403.11722/) (2024)
- [Quaternion-Valued Convolutional Neural Network Applied for Acute Lymphoblastic Leukemia Diagnosis](https://mlsanity.com/paper/2112.06685/) (2021)
- [Enhancing ResNet Image Classification Performance by using Parameterized Hypercomplex Multiplication](https://mlsanity.com/paper/2301.04623/) (2023)
- [Neural Networks at a Fraction with Pruned Quaternions](https://mlsanity.com/paper/2308.06780/) (2023)
- [Quaternion recurrent neural network with real-time recurrent learning and maximum correntropy criterion](https://mlsanity.com/paper/2402.14227/) (2024)
- [Improving Quaternion Neural Networks with Quaternionic Activation Functions](https://mlsanity.com/paper/2406.16481/) (2024)
- [Fully tensorial approach to hypercomplex-valued neural networks](https://mlsanity.com/paper/2407.00449/) (2024)
- [Quaternion-Hadamard Network: A Novel Defense Against Adversarial Attacks with a New Dataset](https://mlsanity.com/paper/2502.10452/) (2025)

## Abstract (arXiv)

The field of deep learning has seen significant advancement in recent years. However, much of the existing work has been focused on real-valued numbers. Recent work has shown that a deep learning system using the complex numbers can be deeper for a fixed parameter budget compared to its real-valued counterpart. In this work, we explore the benefits of generalizing one step further into the hyper-complex numbers, quaternions specifically, and provide the architecture components needed to build deep quaternion networks. We develop the theoretical basis by reviewing quaternion convolutions, developing a novel quaternion weight initialization scheme, and developing novel algorithms for quaternion batch-normalization. These pieces are tested in a classification model by end-to-end training on the CIFAR-10 and CIFAR-100 data sets and a segmentation model by end-to-end training on the KITTI Road Segmentation data set. These quaternion networks show improved convergence compared to real-valued and complex-valued networks, especially on the segmentation task, while having fewer parameters