mlsanityData to 14 Oct 2026

Deep Neural Networks pruning via the Structured Perspective Regularization

arXiv 2206.14056 ↗First posted 28 Jun 2022cs.LG cs.CV math.OCcited by 4

Not on the map. Why?

Abstract (arXiv)

In Machine Learning, Artificial Neural Networks (ANNs) are a very powerful tool, broadly used in many applications. Often, the selected (deep) architectures include many layers, and therefore a large amount of parameters, which makes training, storage and inference expensive. This motivated a stream of research about compressing the original networks into smaller ones without excessively sacrificing performances. Among the many proposed compression approaches, one of the most popular is pruning, whereby entire elements of the ANN (links, nodes, channels, ldots) and the corresponding weights are deleted. Since the nature of the problem is inherently combinatorial (what elements to prune and what not), we propose a new pruning method based on Operational Research tools. We start from a natural Mixed-Integer-Programming model for the problem, and we use the Perspective Reformulation technique to strengthen its continuous relaxation. Projecting away the indicator variables from this reformulation yields a new regularization term, which we call the Structured Perspective Regularization, that leads to structured pruning of the initial architecture. We test our method on some ResNet architectures applied to CIFAR-10, CIFAR-100 and ImageNet datasets, obtaining competitive performances w.r.t.~the state of the art for structured pruning.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 20 of its 40 references that are in the map. First posted on arXiv 28 Jun 2022.

Builds on

20 of 40 references.

References · 20

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  4. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  5. 2016Pruning Filters for Efficient ConvNetscited by 1,143
  6. 2020HRank: Filter Pruning using High-Rank Feature Mapcited by 292
  7. 2017Data-Driven Sparse Structure Selection for Deep Neural Networkscited by 181
  8. 2018Strong mixed-integer programming formulations for trained neural networkscited by 115
  9. 2021CHIP: CHannel Independence-based Pruning for Compact Neural Networkscited by 78
  10. 2019Pruning from Scratchcited by 72
  11. 2021Only Train Once: A One-Shot Neural Network Training And Pruning Frameworkcited by 70
  12. 2020Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformerscited by 67
8 more
  1. 2017Extremely Low Bit Neural Network: Squeeze the Last Bit Out with ADMMcited by 61
  2. 2020A Tunable Robust Pruning Framework Through Dynamic Network Rewiring of DNNscited by 37
  3. 2020Lossless Compression of Deep Neural Networkscited by 34
  4. 2017Deep Neural Networks as 0-1 Mixed Integer Linear Programs: A Feasibility Studycited by 25
  5. 2020Holistic Filter Pruning for Efficient Deep Neural Networkscited by 4
  6. 2020Identifying Critical Neurons in ANN Architectures using Mixed Integer Programmingcited by 3
  7. 2021L2PF -- Learning to Prune Fastercited by 2
  8. 2016Net-Trim: Convex Pruning of Deep Neural Networks with Performance Guaranteecited by 1

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 0; 2024: 0; 2025: 3.

4 papers.

  1. 2022State-driven Implicit Modeling for Sparsity and Robustness in Neural Networkscited by 1
  2. 2025Scalable First-order Method for Certifying Optimal k-Sparse GLMscited by 1
  3. 2025A Metric Topology of Deep Learning for Data Classificationcited by 0
  4. 2025Optimization over Trained (and Sparse) Neural Networks: A Surrogate within a Surrogatecited by 0