# Compressing Quaternion Convolutional Neural Networks for Audio Classification

arXiv: https://arxiv.org/abs/2510.21388 · HTML: https://mlsanity.com/paper/2510.21388/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 47 references that are in the map.
- First posted on arXiv 24 Oct 2025.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [AST: Audio Spectrogram Transformer](https://mlsanity.com/paper/2104.01778/) (2021)
- [PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition](https://mlsanity.com/paper/1912.10211/) (2019)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases](https://mlsanity.com/paper/2103.10697/) (2021)
- [Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration](https://mlsanity.com/paper/1811.00250/) (2018)
- [Inductive Biases for Deep Learning of Higher-Level Cognition](https://mlsanity.com/paper/2011.15091/) (2020)
- [Pruning and Quantization for Deep Neural Network Acceleration: A Survey](https://mlsanity.com/paper/2101.09671/) (2021)
- [Deep Complex Networks](https://mlsanity.com/paper/1705.09792/) (2017)
- [Quaternion Knowledge Graph Embeddings](https://mlsanity.com/paper/1904.10281/) (2019)
- [Structured Pruning for Deep Convolutional Neural Networks: A survey](https://mlsanity.com/paper/2303.00566/) (2023)
- [PSLA: Improving Audio Tagging with Pretraining, Sampling, Labeling, and Aggregation](https://mlsanity.com/paper/2102.01243/) (2021)
- [AutoPruner: An End-to-End Trainable Filter Pruning Method for Efficient Deep Model Inference](https://mlsanity.com/paper/1805.08941/) (2018)
- [Masked Spectrogram Prediction For Self-Supervised Audio Pre-Training](https://mlsanity.com/paper/2204.12768/) (2022)
- [Deep Quaternion Networks](https://mlsanity.com/paper/1712.04604/) (2017)
- [Quaternion Convolutional Neural Networks](https://mlsanity.com/paper/1903.00658/) (2019)
- [ERANNs: Efficient Residual Audio Neural Networks for Audio Pattern Recognition](https://mlsanity.com/paper/2106.01621/) (2021)
- [Efficient Large-scale Audio Tagging via Transformer-to-CNN Knowledge Distillation](https://mlsanity.com/paper/2211.04772/) (2022)
- [PHNNs: Lightweight Neural Networks via Parameterized Hypercomplex Convolutions](https://mlsanity.com/paper/2110.04176/) (2021)
- [Computation-Efficient Knowledge Distillation via Uncertainty-Aware Mixup](https://mlsanity.com/paper/2012.09413/) (2020)
- [Quaternion Convolutional Neural Networks for End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/1806.07789/) (2018)
- [Theoretical Analysis of Inductive Biases in Deep Convolutional Networks](https://mlsanity.com/paper/2305.08404/) (2023)
- [A decade of DCASE: Achievements, practices, evaluations and future challenges](https://mlsanity.com/paper/2410.04951/) (2024)
- [Efficient CNNs via Passive Filter Pruning](https://mlsanity.com/paper/2304.02319/) (2023)
- [A Lottery Ticket Hypothesis Framework for Low-Complexity Device-Robust Neural Acoustic Scene Classification](https://mlsanity.com/paper/2107.01461/) (2021)
- [Pruning vs XNOR-Net: A Comprehensive Study of Deep Learning for Audio Classification on Edge-devices](https://mlsanity.com/paper/2108.06128/) (2021)

## Abstract (arXiv)

Conventional Convolutional Neural Networks (CNNs) in the real domain have been widely used for audio classification. However, their convolution operations process multi-channel inputs independently, limiting the ability to capture correlations among channels. This can lead to suboptimal feature learning, particularly for complex audio patterns such as multi-channel spectrogram representations. Quaternion Convolutional Neural Networks (QCNNs) address this limitation by employing quaternion algebra to jointly capture inter-channel dependencies, enabling more compact models with fewer learnable parameters while better exploiting the multi-dimensional nature of audio signals. However, QCNNs exhibit higher computational complexity due to the overhead of quaternion operations, resulting in increased inference latency and reduced efficiency compared to conventional CNNs, posing challenges for deployment on resource-constrained platforms. To address this challenge, this study explores knowledge distillation (KD) and pruning, to reduce the computational complexity of QCNNs while maintaining performance. Our experiments on audio classification reveal that pruning QCNNs achieves similar or superior performance compared to KD while requiring less computational effort. Compared to conventional CNNs and Transformer-based architectures, pruned QCNNs achieve competitive performance with a reduced learnable parameter count and computational complexity. On the AudioSet dataset, pruned QCNNs reduce computational cost by 50% and parameter count by 80%, while maintaining performance comparable to the conventional CNNs. Furthermore, pruned QCNNs generalize well across multiple audio classification benchmarks, including GTZAN for music genre recognition, ESC-50 for environmental sound classification and RAVDESS for speech emotion recognition.