# Differentiable Mask for Pruning Convolutional and Recurrent Networks

arXiv: https://arxiv.org/abs/1909.04567 · HTML: https://mlsanity.com/paper/1909.04567/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Sep 2019.

## Built on by (most cited first, 10 of 10)

- [Quantization and Deployment of Deep Neural Networks on Microcontrollers](https://mlsanity.com/paper/2105.13331/) (2021)
- [PDP: Parameter-free Differentiable Pruning is All You Need](https://mlsanity.com/paper/2305.11203/) (2023)
- [On the Convergence of Stochastic Gradient Descent in Low-precision Number Formats](https://mlsanity.com/paper/2301.01651/) (2023)
- [iRNN: Integer-only Recurrent Neural Network](https://mlsanity.com/paper/2109.09828/) (2021)
- [A Deeper Look into Convolutions via Eigenvalue-based Pruning](https://mlsanity.com/paper/2102.02804/) (2021)
- [Adaptive Heavy-Tailed Stochastic Gradient Descent](https://mlsanity.com/paper/2508.21353/) (2025)
- [DNN Quantization with Attention](https://mlsanity.com/paper/2103.13322/) (2021)
- [Weight Reparametrization for Budget-Aware Network Pruning](https://mlsanity.com/paper/2107.03909/) (2021)
- [A Twin Neural Model for Uplift](https://mlsanity.com/paper/2105.05146/) (2021)
- [SequentialAttention++ for Block Sparsification: Differentiable Pruning Meets Combinatorial Optimization](https://mlsanity.com/paper/2402.17902/) (2024)

## Abstract (arXiv)

Pruning is one of the most effective model reduction techniques. Deep networks require massive computation and such models need to be compressed to bring them on edge devices. Most existing pruning techniques are focused on vision-based models like convolutional networks, while text-based models are still evolving. The emergence of multi-modal multi-task learning calls for a general method that works on vision and text architectures simultaneously. We introduce a differentiable mask, that induces sparsity on various granularity to fill this gap. We apply our method successfully to prune weights, filters, subnetwork of a convolutional architecture, as well as nodes of a recurrent network.