# DMCP: Differentiable Markov Channel Pruning for Neural Networks

arXiv: https://arxiv.org/abs/2005.03354 · HTML: https://mlsanity.com/paper/2005.03354/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 May 2020.

## Built on by (most cited first, 30 of 69)

- [A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations](https://mlsanity.com/paper/2308.06767/) (2023)
- [Structured Pruning Learns Compact and Accurate Models](https://mlsanity.com/paper/2204.00408/) (2022)
- [Structured Pruning for Deep Convolutional Neural Networks: A survey](https://mlsanity.com/paper/2303.00566/) (2023)
- [CHIP: CHannel Independence-based Pruning for Compact Neural Networks](https://mlsanity.com/paper/2110.13981/) (2021)
- [DiffRate : Differentiable Compression Rate for Efficient Vision Transformers](https://mlsanity.com/paper/2305.17997/) (2023)
- [Hessian-Aware Pruning and Optimal Neural Implant](https://mlsanity.com/paper/2101.08940/) (2021)
- [CHEX: CHannel EXploration for CNN Model Compression](https://mlsanity.com/paper/2203.15794/) (2022)
- [Pruning-as-Search: Efficient Neural Architecture Search via Channel Pruning and Structural Reparameterization](https://mlsanity.com/paper/2206.01198/) (2022)
- [SepPrune: Structured Pruning for Efficient Deep Speech Separation](https://mlsanity.com/paper/2505.12079/) (2025)
- [Topology-Aware Network Pruning using Multi-stage Graph Embedding and Reinforcement Learning](https://mlsanity.com/paper/2102.03214/) (2021)
- [SOSP: Efficiently Capturing Global Correlations by Second-Order Structured Pruning](https://mlsanity.com/paper/2110.11395/) (2021)
- [GDP: Stabilized Neural Network Pruning via Gates with Differentiable Polarization](https://mlsanity.com/paper/2109.02220/) (2021)
- [Locally Free Weight Sharing for Network Width Search](https://mlsanity.com/paper/2102.05258/) (2021)
- [Effective Model Sparsification by Scheduled Grow-and-Prune Methods](https://mlsanity.com/paper/2106.09857/) (2021)
- [VkD: Improving Knowledge Distillation using Orthogonal Projections](https://mlsanity.com/paper/2403.06213/) (2024)
- [BCNet: Searching for Network Width with Bilaterally Coupled Network](https://mlsanity.com/paper/2105.10533/) (2021)
- [PSViT: Better Vision Transformer via Token Pooling and Attention Sharing](https://mlsanity.com/paper/2108.03428/) (2021)
- [Compiler-Aware Neural Architecture Search for On-Mobile Real-time Super-Resolution](https://mlsanity.com/paper/2207.12577/) (2022)
- [Machine Unlearning: A Survey](https://mlsanity.com/paper/2306.03558/) (2023)
- [1xN Pattern for Pruning Convolutional Neural Networks](https://mlsanity.com/paper/2105.14713/) (2021)
- [Trainability Preserving Neural Pruning](https://mlsanity.com/paper/2207.12534/) (2022)
- [Unlearning through Knowledge Overwriting: Reversible Federated Unlearning via Selective Sparse Adapter](https://mlsanity.com/paper/2502.20709/) (2025)
- [Attentive Fine-Grained Structured Sparsity for Image Restoration](https://mlsanity.com/paper/2204.12266/) (2022)
- [PDP: Parameter-free Differentiable Pruning is All You Need](https://mlsanity.com/paper/2305.11203/) (2023)
- [Green Federated Learning: A new era of Green Aware AI](https://mlsanity.com/paper/2409.12626/) (2024)
- [FALCON: FLOP-Aware Combinatorial Optimization for Neural Network Pruning](https://mlsanity.com/paper/2403.07094/) (2024)
- [Lottery Jackpots Exist in Pre-trained Models](https://mlsanity.com/paper/2104.08700/) (2021)
- [Differentiable Transportation Pruning](https://mlsanity.com/paper/2307.08483/) (2023)
- [Compressing Models with Few Samples: Mimicking then Replacing](https://mlsanity.com/paper/2201.02620/) (2022)
- [HRel: Filter Pruning based on High Relevance between Activation Maps and Class Labels](https://mlsanity.com/paper/2202.10716/) (2022)

## Abstract (arXiv)

Recent works imply that the channel pruning can be regarded as searching optimal sub-structure from unpruned networks. However, existing works based on this observation require training and evaluating a large number of structures, which limits their application. In this paper, we propose a novel differentiable method for channel pruning, named Differentiable Markov Channel Pruning (DMCP), to efficiently search the optimal sub-structure. Our method is differentiable and can be directly optimized by gradient descent with respect to standard task loss and budget regularization (e.g. FLOPs constraint). In DMCP, we model the channel pruning as a Markov process, in which each state represents for retaining the corresponding channel during pruning, and transitions between states denote the pruning process. In the end, our method is able to implicitly select the proper number of channels in each layer by the Markov process with optimized transitions. To validate the effectiveness of our method, we perform extensive experiments on Imagenet with ResNet and MobilenetV2. Results show our method can achieve consistent improvement than state-of-the-art pruning methods in various FLOPs settings. The code is available at https://github.com/zx55/dmcp