# Memory Bounded Deep Convolutional Networks

arXiv: https://arxiv.org/abs/1412.1442 · HTML: https://mlsanity.com/paper/1412.1442/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Dec 2014.

## Built on by (most cited first, 17 of 17)

- [Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks](https://mlsanity.com/paper/2102.00554/) (2021)
- [The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation](https://mlsanity.com/paper/2110.03036/) (2021)
- [DECORE: Deep Compression with Reinforcement Learning](https://mlsanity.com/paper/2106.06091/) (2021)
- [Fast, Differentiable and Sparse Top-k: a Convex Analysis Perspective](https://mlsanity.com/paper/2302.01425/) (2023)
- [Data-Efficient Structured Pruning via Submodular Optimization](https://mlsanity.com/paper/2203.04940/) (2022)
- [Neural Network Training Using ℓ1-Regularization and Bi-fidelity Data](https://mlsanity.com/paper/2105.13011/) (2021)
- [Manipulating Identical Filter Redundancy for Efficient Pruning on Deep and Complicated CNN](https://mlsanity.com/paper/2107.14444/) (2021)
- [Spartan: Differentiable Sparsity via Regularized Transportation](https://mlsanity.com/paper/2205.14107/) (2022)
- [Sum-Rate-Distortion Function for Indirect Multiterminal Source Coding in Federated Learning](https://mlsanity.com/paper/2101.08696/) (2021)
- [A phase transition for finding needles in nonlinear haystacks with LASSO artificial neural networks](https://mlsanity.com/paper/2201.08652/) (2022)
- [Model Compression Methods for YOLOv5: A Review](https://mlsanity.com/paper/2307.11904/) (2023)
- [Differentiable Architecture Pruning for Transfer Learning](https://mlsanity.com/paper/2107.03375/) (2021)
- [Sparse Deep Learning Models with the ℓ1 Regularization](https://mlsanity.com/paper/2408.02801/) (2024)
- [Neural Network Reduction with Guided Regularizers](https://mlsanity.com/paper/2305.18448/) (2023)
- [Generalization Bounds for Magnitude-Based Pruning via Sparse Matrix Sketching](https://mlsanity.com/paper/2305.18789/) (2023)
- [Efficient Model Compression for Bayesian Neural Networks](https://mlsanity.com/paper/2411.00273/) (2024)
- [Coding for Computation: Efficient Compression of Neural Networks for Reconfigurable Hardware](https://mlsanity.com/paper/2504.17403/) (2025)

## Abstract (arXiv)

In this work, we investigate the use of sparsity-inducing regularizers during training of Convolution Neural Networks (CNNs). These regularizers encourage that fewer connections in the convolution and fully connected layers take non-zero values and in effect result in sparse connectivity between hidden units in the deep network. This in turn reduces the memory and runtime cost involved in deploying the learned CNNs. We show that training with such regularization can still be performed using stochastic gradient descent implying that it can be used easily in existing codebases. Experimental evaluation of our approach on MNIST, CIFAR, and ImageNet datasets shows that our regularizers can result in dramatic reductions in memory requirements. For instance, when applied on AlexNet, our method can reduce the memory consumption by a factor of four with minimal loss in accuracy.