# ProdSumNet: reducing model parameters in deep neural networks via product-of-sums matrix decompositions

arXiv: https://arxiv.org/abs/1809.02209 · HTML: https://mlsanity.com/paper/1809.02209/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Sep 2018.

## Built on by (most cited first, 5 of 5)

- [WaveMix: A Resource-efficient Neural Network for Image Analysis](https://mlsanity.com/paper/2205.14375/) (2022)
- [LightLayers: Parameter Efficient Dense and Convolutional Layers for Image Classification](https://mlsanity.com/paper/2101.02268/) (2021)
- [Context-Selective State Space Models: Feedback is All You Need](https://mlsanity.com/paper/2510.14027/) (2025)
- [STen: Productive and Efficient Sparsity in PyTorch](https://mlsanity.com/paper/2304.07613/) (2023)
- [LTBs-KAN: Linear-Time B-splines Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2604.22034/) (2026)

## Abstract (arXiv)

We consider a general framework for reducing the number of trainable model parameters in deep learning networks by decomposing linear operators as a product of sums of simpler linear operators. Recently proposed deep learning architectures such as CNN, KFC, Dilated CNN, etc. are all subsumed in this framework and we illustrate other types of neural network architectures within this framework. We show that good accuracy on MNIST and Fashion MNIST can be obtained using a relatively small number of trainable parameters. In addition, since implementation of the convolutional layer is resource-heavy, we consider an approach in the transform domain that obviates the need for convolutional layers. One of the advantages of this general framework over prior approaches is that the number of trainable parameters is not fixed and can be varied arbitrarily. In particular, we illustrate the tradeoff of varying the number of trainable variables and the corresponding error rate. As an example, by using this decomposition on a reference CNN architecture for MNIST with over 3x10^6 trainable parameters, we are able to obtain an accuracy of 98.44% using only 3554 trainable parameters.