mlsanityData to 14 Oct 2026

FAIR-Pruner: A Flexible Framework for Automatic Layer-Wise Pruning via Tolerance of Difference

arXiv 2508.02291 ↗First posted 4 Aug 2025cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Structured pruning is a standard tool for compressing deep neural networks, but its practical performance depends on how sparsity is allocated across layers. We propose FAIR-Pruner, a search-free framework for adaptive layer-wise structured pruning. FAIR-Pruner uses two within-layer rankings: a removal-oriented signal that proposes candidate units and a protection-oriented signal that identifies task-sensitive units. Its core component, Tolerance of Difference (ToD), measures the overlap between the removal prefix and the protected tail, and uses a shared tolerance level to induce non-uniform pruning depths across layers. As a default vision instantiation, FAIR-Pruner combines a Wasserstein-based U-Score for class-conditional unit separability with a Taylor-based R-Score for task-level sensitivity; the same ToD allocation rule can also be paired with alternative removal signals. Theoretically, we analyze ToD through the population R-Score, derive rank-based control of the high-R-Score mass entering the pruning set, and identify an additive exchange condition for same-budget comparison with uniform pruning. Experiments on CIFAR-10, CIFAR-100, SVHN, and ImageNet across VGG, ResNet, DenseNet, ConvNeXt, and DeiT show strong accuracy--compression trade-offs. Prune-only experiments on routed-expert Qwen1.5-MoE-A2.7B-Chat further examine architectural extensibility under matched expert budgets. FAIR-Pruner is released as a pip-installable open-source package.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 39 of its 57 references that are in the map. First posted on arXiv 4 Aug 2025.

Builds on

39 of 57 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2016Densely Connected Convolutional Networkscited by 6,720
  4. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  5. 2022A ConvNet for the 2020scited by 3,579
  6. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  7. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  8. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  9. 2016Pruning Filters for Efficient ConvNetscited by 1,143
  10. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 702
  11. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 676
  12. 2017Learning Sparse Neural Networks through L0 Regularizationcited by 583
27 more
  1. 2020What is the State of Neural Network Pruning?cited by 583
  2. 2019Importance Estimation for Neural Network Pruningcited by 541
  3. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 378
  4. 2018WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representationscited by 372
  5. 2018Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Accelerationcited by 367
  6. 2018Soft Filter Pruning for Accelerating Deep Convolutional Neural Networkscited by 315
  7. 2020HRank: Filter Pruning using High-Rank Feature Mapcited by 292
  8. 2019MetaPruning: Meta Learning for Automatic Neural Network Channel Pruningcited by 227
  9. 2017Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeoncited by 221
  10. 2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 202
  11. 2019Towards Optimal Structured CNN Pruning via Generative Adversarial Learningcited by 146
  12. 2021IA-RED2: Interpretability-Aware Redundancy Reduction for Vision Transformerscited by 127
  13. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  14. 2018Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layerscited by 112
  15. 2019EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasiscited by 98
  16. 2020Neural Pruning via Growing Regularizationcited by 97
  17. 2019Pruning by Explaining: A Novel Criterion for Deep Neural Network Pruningcited by 89
  18. 2019Network Pruning via Transformable Architecture Searchcited by 80
  19. 2020DMCP: Differentiable Markov Channel Pruning for Neural Networkscited by 69
  20. 2022Revisiting Random Channel Pruning for Neural Network Compressioncited by 50
  21. 2024SEER-MoE: Sparse Expert Efficiency through Regularization for Mixture-of-Expertscited by 40
  22. 2021DECORE: Deep Compression with Reinforcement Learningcited by 29
  23. 2020Auto Graph Encoder-Decoder for Neural Network Pruningcited by 26
  24. 2021An Information Theory-inspired Strategy for Automatic Network Pruningcited by 17
  25. 2019Filter Pruning by Switching to Neighboring CNNs with Good Attributescited by 15
  26. 2020CPOT: Channel Pruning via Optimal Transportcited by 5
  27. 2023SWAP: Sparse Entropic Wasserstein Regression for Robust Network Pruningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024RL-Pruner: Structured Pruning Using Reinforcement Learning for CNN Compression and Acceleration
  2. 2025GoPrune: Accelerated Structured Pruning with ℓ2,p-Norm Optimization
  3. 2022CAP: Correlation-Aware Pruning for Highly-Accurate Sparse Vision Models
  4. 2021Dynamic Probabilistic Pruning: A general framework for hardware-constrained pruning at different granularities
  5. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Survey
  6. 2024Comb, Prune, Distill: Towards Unified Pruning for Vision Model Compression
  7. 2022Boosting Pruned Networks with Linear Over-parameterization
  8. 2022SPDY: Accurate Pruning with Speedup Guarantees