# Compressing Neural Networks Using Tensor Networks with Exponentially Fewer Variational Parameters

arXiv: https://arxiv.org/abs/2305.06058 · HTML: https://mlsanity.com/paper/2305.06058/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 0 of its 1 references that are in the map.
- First posted on arXiv 10 May 2023.

## Built on by (most cited first, 9 of 9)

- [Tensor networks for interpretable and efficient quantum-inspired machine learning](https://mlsanity.com/paper/2311.11258/) (2023)
- [Edge AI: Evaluation of Model Compression Techniques for Convolutional Neural Networks](https://mlsanity.com/paper/2409.02134/) (2024)
- [Quantum-enhanced unsupervised image segmentation for medical images analysis](https://mlsanity.com/paper/2411.15086/) (2024)
- [Compressing Transformer Language Models via Matrix Product Operator Decomposition: A Case Study on PicoGPT](https://mlsanity.com/paper/2603.28534/) (2026)
- [Quantum-inspired Techniques in Tensor Networks for Industrial Contexts](https://mlsanity.com/paper/2404.11277/) (2024)
- [Efficient Finite Initialization with Partial Norms for Tensorized Neural Networks and Tensor Networks Algorithms](https://mlsanity.com/paper/2309.06577/) (2023)
- [T'ecnicas Quantum-Inspired en Tensor Networks para Contextos Industriales](https://mlsanity.com/paper/2404.17645/) (2024)
- [Universal replication of chaotic characteristics by classical and quantum machine learning](https://mlsanity.com/paper/2405.08484/) (2024)
- [A Hamiltonian-Inspired Local-Operator Ansatz for Slimming Large Language Models](https://mlsanity.com/paper/2605.25344/) (2026)

## Abstract (arXiv)

Neural network (NN) designed for challenging machine learning tasks is in general a highly nonlinear mapping that contains massive variational parameters. High complexity of NN, if unbounded or unconstrained, might unpredictably cause severe issues including Roverfitting, loss of generalization power, and unbearable cost of hardware. In this work, we propose a general compression scheme that significantly reduces the variational parameters of NN's, despite of their specific types (linear, convolutional, textitetc), by encoding them to deep Rautomatically differentiable tensor network (ADTN) that contains exponentially-fewer free parameters. Superior compression performance of our scheme is demonstrated on several widely-recognized NN's (FC-2, LeNet-5, AlextNet, ZFNet and VGG-16) and datasets (MNIST, CIFAR-10 and CIFAR-100). For instance, we compress two linear layers in VGG-16 with approximately 107 parameters to two ADTN's with just 424 parameters, improving the testing accuracy on CIFAR-10 from 90.17% to 91.74%. We argue that the deep structure of ADTN is an essential reason for the remarkable compression performance of ADTN, compared to existing compression schemes that are mainly based on tensor decompositions/factorization and shallow tensor networks. Our work suggests deep TN as an exceptionally efficient mathematical structure for representing the variational parameters of NN's, which exhibits superior compressibility over the commonly-used matrices and multi-way arrays.