# Building Efficient Deep Neural Networks with Unitary Group Convolutions

arXiv: https://arxiv.org/abs/1811.07755 · HTML: https://mlsanity.com/paper/1811.07755/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2018.

## Built on by (most cited first, 9 of 9)

- [Learning from Students: Applying t-Distributions to Explore Accurate and Efficient Formats for LLMs](https://mlsanity.com/paper/2405.03103/) (2024)
- [Fast Walsh-Hadamard Transform and Smooth-Thresholding Based Binary Layers in Deep Neural Networks](https://mlsanity.com/paper/2104.07085/) (2021)
- [Enabling Design Methodologies and Future Trends for Edge AI: Specialization and Co-design](https://mlsanity.com/paper/2103.15750/) (2021)
- [A compact butterfly-style silicon photonic-electronic neural chip for hardware-efficient deep learning](https://mlsanity.com/paper/2111.06705/) (2021)
- [Block Walsh-Hadamard Transform Based Binary Layers in Deep Neural Networks](https://mlsanity.com/paper/2201.02711/) (2022)
- [RingCNN: Exploiting Algebraically-Sparse Ring Tensors for Energy-Efficient CNN-Based Computational Imaging](https://mlsanity.com/paper/2104.09056/) (2021)
- [XCAT -- Lightweight Quantized Single Image Super-Resolution using Heterogeneous Group Convolutions and Cross Concatenation](https://mlsanity.com/paper/2208.14655/) (2022)
- [Boosting Convolutional Neural Networks with Middle Spectrum Grouped Convolution](https://mlsanity.com/paper/2304.06305/) (2023)
- [Beyond Oversquashing: Understanding Signal Propagation in GNNs Via Observables](https://mlsanity.com/paper/2605.13383/) (2026)

## Abstract (arXiv)

We propose unitary group convolutions (UGConvs), a building block for CNNs which compose a group convolution with unitary transforms in feature space to learn a richer set of representations than group convolution alone. UGConvs generalize two disparate ideas in CNN architecture, channel shuffling (i.e. ShuffleNet) and block-circulant networks (i.e. CirCNN), and provide unifying insights that lead to a deeper understanding of each technique. We experimentally demonstrate that dense unitary transforms can outperform channel shuffling in DNN accuracy. On the other hand, different dense transforms exhibit comparable accuracy performance. Based on these observations we propose HadaNet, a UGConv network using Hadamard transforms. HadaNets achieve similar accuracy to circulant networks with lower computation complexity, and better accuracy than ShuffleNets with the same number of parameters and floating-point multiplies.