Unified Response Geometry for Structured Pruning
Not on the map. Why?
Abstract (arXiv)
Structured pruning is commonly formulated as ranking individual channels, although channel responses can be complementary or cancel through downstream mixing. Motivated by these response interactions, we formulate pruning as the selection of a subset with large joint response capacity, followed by a separate functional realization step. Our unified response geometry maps each candidate set to (M(D,R)=D1/2RD1/2) and uses its determinant together with Schur-greedy residuals to select non-redundant coordinates. The same construction yields two information-conditioned instances: an unlabeled instance based on activation covariance, and a task-conditioned instance that combines activation and gradient variance for response scale with gradient correlation for complementarity. To convert the selected subset into an executable network, we fold predictable removed responses into successor weights through ridge compensation and recalibrate batch-normalization statistics, without fine-tuning the network. On ImageNet ResNet-50, the unlabeled instance reaches (65.4%) and (53.9%) Top-1 accuracy at 30% and 40% deletion, versus (59.8%) and (43.1%) for strength-only selection; the task-conditioned instance reaches (67.7%) and (56.3%) under the same protocol. A six-family screen shows architecture-dependent behavior, with positive relative contrasts in several convolutional and expansion-layer settings and clear boundary cases in windowed attention. These results support response geometry as a conditional principle for structured pruning, with its benefit determined jointly by the observed response and the architecture in which that response is realized.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 34 of its 39 references that are in the map. First posted on arXiv 16 Sep 2026.
Builds on
34 of 39 references.
References · 34
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
- 2022A ConvNet for the 2020scited by 3,579
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
- 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
- 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
- 2016Pruning Filters for Efficient ConvNetscited by 1,143
- 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
- 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 812
22 more
- 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
- 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 702
- 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 676
- 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 650
- 2018Rethinking the Value of Network Pruningcited by 599
- 2016Learning Structured Sparsity in Deep Neural Networkscited by 541
- 2019Importance Estimation for Neural Network Pruningcited by 541
- 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 479
- 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 471
- 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 442
- 2020Movement Pruning: Adaptive Sparsity by Fine-Tuningcited by 403
- 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 378
- 2018Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Accelerationcited by 367
- 2018Soft Filter Pruning for Accelerating Deep Convolutional Neural Networkscited by 315
- 2020HRank: Filter Pruning using High-Rank Feature Mapcited by 292
- 2017NISP: Pruning Networks using Neuron Importance Score Propagationcited by 228
- 2019MetaPruning: Meta Learning for Automatic Neural Network Channel Pruningcited by 227
- 2016Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architecturescited by 221
- 2022Structured Pruning Learns Compact and Accurate Modelscited by 170
- 2018NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applicationscited by 147
- 2020Layer-adaptive sparsity for the Magnitude-based Pruningcited by 126
- 2026Adjoint Inversion Reveals Holographic Superposition and Destructive Interference in CNN Classifierscited by 2
Built on by
Not cited yet.