# Two-Valued Symmetric Circulant Matrices: Applications in Deep Learning

arXiv: https://arxiv.org/abs/2605.16443 · HTML: https://mlsanity.com/paper/2605.16443/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 62 references that are in the map.
- First posted on arXiv 15 May 2026.

## Builds on (most cited first, 18 of 18)

- [Rethinking the Value of Network Pruning](https://mlsanity.com/paper/1810.05270/) (2018)
- [What is the State of Neural Network Pruning?](https://mlsanity.com/paper/2003.03033/) (2020)
- [Monarch: Expressive Structured Matrices for Efficient and Accurate Training](https://mlsanity.com/paper/2204.00595/) (2022)
- [CirCNN: Accelerating and Compressing Deep Neural Networks Using Block-CirculantWeight Matrices](https://mlsanity.com/paper/1708.08917/) (2017)
- [Structured Transforms for Small-Footprint Deep Learning](https://mlsanity.com/paper/1510.01722/) (2015)
- [A Deep Learning approach to Reduced Order Modelling of Parameter Dependent Partial Differential Equations](https://mlsanity.com/paper/2103.06183/) (2021)
- [Domain Generalization by Learning and Removing Domain-specific Features](https://mlsanity.com/paper/2212.07101/) (2022)
- [Trained Rank Pruning for Efficient Deep Neural Networks](https://mlsanity.com/paper/1812.02402/) (2018)
- [Domain-Specific Risk Minimization for Out-of-Distribution Generalization](https://mlsanity.com/paper/2208.08661/) (2022)
- [PERMDNN: Efficient Compressed DNN Architecture with Permuted Diagonal Matrices](https://mlsanity.com/paper/2004.10936/) (2020)
- [BlockGNN: Towards Efficient GNN Acceleration Using Block-Circulant Weight Matrices](https://mlsanity.com/paper/2104.06214/) (2021)
- [CircConv: A Structured Convolution with Low Complexity](https://mlsanity.com/paper/1902.11268/) (2019)
- [Edge Intelligence for Empowering IoT-based Healthcare Systems](https://mlsanity.com/paper/2103.12144/) (2021)
- [Structured Unrestricted-Rank Matrices for Parameter Efficient Fine-tuning](https://mlsanity.com/paper/2406.17740/) (2024)
- [Understanding and Training Deep Diagonal Circulant Neural Networks](https://mlsanity.com/paper/1901.10255/) (2019)
- [Structure-Preserving Network Compression Via Low-Rank Induced Training Through Linear Layers Composition](https://mlsanity.com/paper/2405.03089/) (2024)
- [Block Circulant Adapter for Large Language Models](https://mlsanity.com/paper/2505.00582/) (2025)
- [Classification of Informative Frames in Colonoscopy Videos Using Convolutional Neural Networks with Binarized Weights](https://mlsanity.com/paper/1802.01387/) (2018)

## Abstract (arXiv)

Despite the success of deep neural networks in vision, medical diagnosis, and IoT scenarios, their deployment on resource-limited platforms poses serious challenges due to their high storage requirements, computational complexity, and large footprint. In particular, fully connected layers require a large number of weights, making it difficult for edge devices to accommodate them. To overcome these challenges associated with limited platforms, this paper proposes the Two-Valued Symmetric Circulant Matrix (TVSCM), a very sparse architecture that employs just two weights per layer to keep it circulant and symmetric. The extreme form of structured sparse architecture provides negligible storage costs compared to traditional full-weight storage. Instead of hardware and additional stages of other traditional sparse learning techniques, such as low-rank approximation and pruning approaches, this architecture provides an extreme form of sparsity, achieving very minimal storage requirements. The simulation study demonstrates more than 80× reduction in model parameters, reducing parameters from 623,290 to 7,852 on MNIST and from 24,709 to 942 on the MIT-BIH arrhythmia dataset, while maintaining comparable accuracy from 97.6% to 93.5% on MNIST and from 97.6% to 93.1% on MIT-BIH. Due to its minimal architectural requirements and very low power consumption, this architecture would be ideal for edge computing platforms, tiny-ML platforms, IoMT systems, and battery-powered systems.