# Spectral Pruning: Compressing Deep Neural Networks via Spectral Analysis and its Generalization Error

arXiv: https://arxiv.org/abs/1808.08558 · HTML: https://mlsanity.com/paper/1808.08558/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Aug 2018.

## Built on by (most cited first, 4 of 4)

- [The Combinatorial Brain Surgeon: Pruning Weights That Cancel One Another in Neural Networks](https://mlsanity.com/paper/2203.04466/) (2022)
- [Scaling Up Exact Neural Network Compression by ReLU Stability](https://mlsanity.com/paper/2102.07804/) (2021)
- [Heating up decision boundaries: isocapacitory saturation, adversarial scenarios and generalization bounds](https://mlsanity.com/paper/2101.06061/) (2021)
- [PrunedCaps: A Case For Primary Capsules Discrimination](https://mlsanity.com/paper/2512.06003/) (2025)

## Abstract (arXiv)

Compression techniques for deep neural network models are becoming very important for the efficient execution of high-performance deep learning systems on edge-computing devices. The concept of model compression is also important for analyzing the generalization error of deep learning, known as the compression-based error bound. However, there is still huge gap between a practically effective compression method and its rigorous background of statistical learning theory. To resolve this issue, we develop a new theoretical framework for model compression and propose a new pruning method called it spectral pruning based on this framework. We define the ``degrees of freedom'' to quantify the intrinsic dimensionality of a model by using the eigenvalue distribution of the covariance matrix across the internal nodes and show that the compression ability is essentially controlled by this quantity. Moreover, we present a sharp generalization error bound of the compressed model and characterize the bias--variance tradeoff induced by the compression procedure. We apply our method to several datasets to justify our theoretical analyses and show the superiority of the the proposed method.