mlsanityData to 7 Oct 2026

Differentiable Transportation Pruning

arXiv 2307.08483 ↗First posted 17 Jul 2023cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

Deep learning algorithms are increasingly employed at the edge. However, edge devices are resource constrained and thus require efficient deployment of deep neural networks. Pruning methods are a key tool for edge deployment as they can improve storage, compute, memory bandwidth, and energy usage. In this paper we propose a novel accurate pruning technique that allows precise control over the output network size. Our method uses an efficient optimal transportation scheme which we make end-to-end differentiable and which automatically tunes the exploration-exploitation behavior of the algorithm to find accurate sparse sub-networks. We show that our method achieves state-of-the-art performance compared to previous pruning methods on 3 different datasets, using 5 different models, across a wide range of pruning ratios, and with two types of sparsity budgets and pruning granularities.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 47 of its 69 references that are in the map. First posted on arXiv 17 Jul 2023.

Builds on

47 of 69 references.

References · 47

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,277
  4. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  5. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  6. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  7. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
  8. 2018Rethinking the Value of Network Pruningcited by 595
  9. 2019Importance Estimation for Neural Network Pruningcited by 531
  10. 2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 521
  11. 2019The State of Sparsity in Deep Neural Networkscited by 471
  12. 2018Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Accelerationcited by 365
35 more
  1. 2018Soft Filter Pruning for Accelerating Deep Convolutional Neural Networkscited by 313
  2. 2017Learning Generative Models with Sinkhorn Divergencescited by 310
  3. 2020HRank: Filter Pruning using High-Rank Feature Mapcited by 291
  4. 2017Variational Dropout Sparsifies Deep Neural Networkscited by 265
  5. 2019MetaPruning: Meta Learning for Automatic Neural Network Channel Pruningcited by 226
  6. 2017Data-Driven Sparse Structure Selection for Deep Neural Networkscited by 181
  7. 2018Discrimination-aware Channel Pruning for Deep Neural Networkscited by 171
  8. 2020Dynamic Model Pruning with Feedbackcited by 125
  9. 2018Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layerscited by 112
  10. 2019Winning the Lottery with Continuous Sparsificationcited by 99
  11. 2019Provable Filter Pruning for Efficient Neural Networkscited by 98
  12. 2020Neural Pruning via Growing Regularizationcited by 97
  13. 2020EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruningcited by 80
  14. 2018AutoPruner: An End-to-End Trainable Filter Pruning Method for Efficient Deep Model Inferencecited by 77
  15. 2020Operation-Aware Soft Channel Pruning using Differentiable Maskscited by 74
  16. 2021Convolutional Neural Network Pruning with Structural Redundancy Reductioncited by 73
  17. 2015Scalable Gradient-Based Tuning of Continuous Regularization Hyperparameterscited by 71
  18. 2019Global Sparse Momentum SGD for Pruning Very Deep Neural Networkscited by 70
  19. 2016Stabilized Sparse Scaling Algorithms for Entropy Regularized Transport Problemscited by 69
  20. 2020DMCP: Differentiable Markov Channel Pruning for Neural Networkscited by 68
  21. 2019Neural Network Pruning with Residual-Connections and Limited-Datacited by 52
  22. 2022Revisiting Random Channel Pruning for Neural Network Compressioncited by 50
  23. 2020DSA: More Efficient Budgeted Pruning via Differentiable Sparsity Allocationcited by 47
  24. 2020Pruning Filter in Filtercited by 46
  25. 2022Structural Pruning via Latency-Saliency Knapsackcited by 40
  26. 2022A Unified Framework for Implicit Sinkhorn Differentiationcited by 28
  27. 2018Structured Pruning of Neural Networks with Budget-Aware Regularizationcited by 26
  28. 2021On Lottery Tickets and Minimal Task Representations in Deep Reinforcement Learningcited by 23
  29. 2021ChipNet: Budget-Aware Pruning with Heaviside Continuous Approximationscited by 21
  30. 2021Pruning Randomly Initialized Neural Networks with Iterative Randomizationcited by 20
  31. 2022Soft Masking for Cost-Constrained Channel Pruningcited by 12
  32. 2022Spartan: Differentiable Sparsity via Regularized Transportationcited by 8
  33. 2020CPOT: Channel Pruning via Optimal Transportcited by 5
  34. 2022Auto-Compressing Subset Pruning for Semantic Image Segmentationcited by 4
  35. 2017The Power of Sparsity in Convolutional Neural Networkscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 5; 2026: 2.

10 papers.

  1. 2024Auto-Train-Once: Controller Network Guided Automatic Network Pruning from Scratchcited by 20
  2. 2024Jointly Training and Pruning CNNs via Learnable Agent Guidance and Alignmentcited by 9
  3. 2025MDP: Multidimensional Vision Model Pruning with Latency Constraintcited by 4
  4. 2025B-FPGM: Lightweight Face Detection via Bayesian-Optimized Soft FPGM Pruningcited by 2
  5. 2025One-Cycle Structured Pruning via Stability-Driven Subnetwork Searchcited by 1
  6. 2025Meta Pruning via Graph Metanetworks : A Universal Meta Learning Framework for Network Pruningcited by 1
  7. 2024S2HPruner: Soft-to-Hard Distillation Bridges the Discretization Gap in Pruningcited by 0
  8. 2025Neural expressiveness for beyond importance model compressioncited by 0
  9. 2026Spectral Complex Autoencoder Pruning: A Fidelity-Guided Criterion for Extreme Structured Channel Compressioncited by 0
  10. 2026Zero-Inflated Gaussian Distributions Enable Parameter-Space Sparsity in Estimation-of-Distribution Algorithmscited by 0