# Compressing Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1506.04449 · HTML: https://mlsanity.com/paper/1506.04449/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jun 2015.

## Built on by (most cited first, 25 of 25)

- [EAGLES: Efficient Accelerated 3D Gaussians with Lightweight EncodingS](https://mlsanity.com/paper/2312.04564/) (2023)
- [Fast Vision Transformers with HiLo Attention](https://mlsanity.com/paper/2205.13213/) (2022)
- [Towards Discriminative Representation Learning for Unsupervised Person Re-identification](https://mlsanity.com/paper/2108.03439/) (2021)
- [Transform Once: Efficient Operator Learning in Frequency Domain](https://mlsanity.com/paper/2211.14453/) (2022)
- [Computation-efficient Deep Learning for Computer Vision: A Survey](https://mlsanity.com/paper/2308.13998/) (2023)
- [Few-Shot Learning by Integrating Spatial and Frequency Representation](https://mlsanity.com/paper/2105.05348/) (2021)
- [Boosting Night-time Scene Parsing with Learnable Frequency](https://mlsanity.com/paper/2208.14241/) (2022)
- [LilNetX: Lightweight Networks with EXtreme Model Compression and Structured Sparsification](https://mlsanity.com/paper/2204.02965/) (2022)
- [EZCrop: Energy-Zoned Channels for Robust Output Pruning](https://mlsanity.com/paper/2105.03679/) (2021)
- [Model Pruning Based on Quantified Similarity of Feature Maps](https://mlsanity.com/paper/2105.06052/) (2021)
- [On the Robustness of CountSketch to Adaptive Inputs](https://mlsanity.com/paper/2202.13736/) (2022)
- [ADC/DAC-Free Analog Acceleration of Deep Neural Networks with Frequency Transformation](https://mlsanity.com/paper/2309.01771/) (2023)
- [Constrained Multiview Representation for Self-supervised Contrastive Learning](https://mlsanity.com/paper/2402.03456/) (2024)
- [Tricking the Hashing Trick: A Tight Lower Bound on the Robustness of CountSketch to Adaptive Inputs](https://mlsanity.com/paper/2207.00956/) (2022)
- [From Algorithm to Hardware: A Survey on Efficient and Safe Deployment of Deep Neural Networks](https://mlsanity.com/paper/2405.06038/) (2024)
- [AdaSpring: Context-adaptive and Runtime-evolutionary Deep Model Compression for Mobile Applications](https://mlsanity.com/paper/2101.11800/) (2021)
- [ORC: Network Group-based Knowledge Distillation using Online Role Change](https://mlsanity.com/paper/2206.01186/) (2022)
- [Frequency Regularization: Restricting Information Redundancy of Convolutional Neural Networks](https://mlsanity.com/paper/2304.07973/) (2023)
- [Secure and Storage-Efficient Deep Learning Models for Edge AI Using Automatic Weight Generation](https://mlsanity.com/paper/2507.06380/) (2025)
- [Neural Network Pruning as Spectrum Preserving Process](https://mlsanity.com/paper/2307.08982/) (2023)
- [Holistic Geometric Feature Learning for Structured Reconstruction](https://mlsanity.com/paper/2309.09622/) (2023)
- [Containing Analog Data Deluge at Edge through Frequency-Domain Compression in Collaborative Compute-in-Memory Networks](https://mlsanity.com/paper/2309.11048/) (2023)
- [Evaluating Adversarial Robustness in the Spatial Frequency Domain](https://mlsanity.com/paper/2405.06345/) (2024)
- [Communication-Efficient Distributed Training for Collaborative Flat Optima Recovery in Deep Learning](https://mlsanity.com/paper/2507.20424/) (2025)
- [MO-SAE:Multi-Objective Stacked Autoencoders Optimization for Edge Anomaly Detection](https://mlsanity.com/paper/2603.13895/) (2026)

## Abstract (arXiv)

Convolutional neural networks (CNN) are increasingly used in many areas of computer vision. They are particularly attractive because of their ability to "absorb" great quantities of labeled data through millions of parameters. However, as model sizes increase, so do the storage and memory requirements of the classifiers. We present a novel network architecture, Frequency-Sensitive Hashed Nets (FreshNets), which exploits inherent redundancy in both convolutional layers and fully-connected layers of a deep learning model, leading to dramatic savings in memory and storage consumption. Based on the key observation that the weights of learned convolutional filters are typically smooth and low-frequency, we first convert filter weights to the frequency domain with a discrete cosine transform (DCT) and use a low-cost hash function to randomly group frequency parameters into hash buckets. All parameters assigned the same hash bucket share a single value learned with standard back-propagation. To further reduce model size we allocate fewer hash buckets to high-frequency components, which are generally less important. We evaluate FreshNets on eight data sets, and show that it leads to drastically better compressed performance than several relevant baselines.