Relative Kinetic Utility for Reasoning-Aware Structural Pruning in Large Language Models
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
Global structured pruning requires channels from different layers to compete under a shared sparsity budget, raising two coupled challenges: identifying which channels should be retained and making their scores comparable across layers. Raw channel scores can contain block-common scale that leaves within-block ordering unchanged but distorts model-wide competition. Our experiment indicates that similar layer-wise allocations can retain substantially different FFN channels, so layer allocation alone does not determine channel identity. Motivated by this separation, we introduce Global Relative Kinetic Utility (Global RKU), a label-free criterion that separates channel importance estimation from cross-layer comparison. Global RKU measures channel participation using a final-hidden-state activation-gradient signal, then applies block-relative normalization to mitigate block-common scale while preserving within-block ordering, requires only unlabeled calibration inputs, and produces a static pruning topology in a single calibration stage. Under questions-only calibration on Qwen-2.5-7B, RKU-GISP Mean3 margins are -0.98, +3.79, and +8.61 points at 30%, 40%, and 50% sparsity, respectively (average +3.81). Additional Qwen evaluations cover non-mathematical reasoning, recovery, held-out transfer, and physical deployment. Separately, replacing Wiki16K with questions-only Q16K improves RKU's Mean3 at every tested sparsity on Qwen, Llama, and Gemma. Our ablation study shows relative-normalization gains of 14.42 and 5.53 Mean3 points at 40% and 50% sparsity, respectively; the common-seed audit is positive in all 27 seed-task comparisons.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 42 of its 46 references that are in the map. First posted on arXiv 9 May 2026.
Builds on
42 of 46 references.
Closest ancestors · 4
Same direction, newest first
- 2026From LLMs to LRMs: Rethinking Pruning for Reasoning-Centric Models26 Jan
- 2025Think Before You Prune: Self-Reflective Structured Pruning for Reasoning Language Models1 Dec
- 2025Think Before You Prune: Selective Self-Generated Calibration for Pruning Large Reasoning Models24 Nov
- 2024Rethinking Pruning Large Language Models: Benefits and Pitfalls of Reconstruction Error Minimization21 Jun
Other references · 38
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
- 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 844
- 2019Are Sixteen Heads Really Better than One?cited by 803
- 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 802
- 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
- 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
- 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 643
- 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
- 2019Importance Estimation for Neural Network Pruningcited by 531
- 2023Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruningcited by 408
- 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 378
26 more
- 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expectcited by 369
- 2021Block Pruning For Faster Transformerscited by 176
- 2022Structured Pruning Learns Compact and Accurate Modelscited by 167
- 2023Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsitycited by 163
- 2023Fluctuation-based Adaptive Structured Pruning for Large Language Modelscited by 110
- 2023LoRAPrune: Structured Pruning Meets Low-Rank Parameter-Efficient Fine-Tuningcited by 53
- 2023ZipLM: Inference-Aware Structured Pruning of Language Modelscited by 46
- 2024BESA: Pruning Large Language Models with Blockwise Parameter-Efficient Sparsity Allocationcited by 44
- 2024DISP-LLM: Dimension-Independent Structural Pruning for Large Language Modelscited by 24
- 2024SparseLLM: Towards Global Pruning for Pre-trained Language Modelscited by 23
- 2024Is C4 Dataset Optimal for Pruning? An Investigation of Calibration Data for LLM Pruningcited by 19
- 2025SlimLLM: Accurate Structured Pruning for Large Language Modelscited by 18
- 2025DarwinLM: Evolutionary Structured Pruning of Large Language Modelscited by 16
- 2024Beware of Calibration Data for Pruning Large Language Modelscited by 14
- 2025Instruction-Following Pruning for Large Language Modelscited by 14
- 2025Probe Pruning: Accelerating LLMs through Dynamic Pruning via Model-Probingcited by 14
- 2025Alternating Gradient Flows: A Theory of Feature Learning in Two-layer Neural Networkscited by 11
- 2024On the Limitations of Language Targeted Pruning: Investigating the Calibration Language Impact in Multilingual LLM Pruningcited by 6
- 2025Olica: Efficient Structured Pruning of Large Language Models without Retrainingcited by 6
- 2025NIRVANA: Structured Pruning Reimagined for Large Language Model Compressioncited by 6
- 2025From Local to Global: Revisiting Structured Pruning Paradigms for Large Language Modelscited by 5
- 2025RCPU: Rotation-Constrained Error Compensation for Structured Pruning of Large Language Modelscited by 2
- 2025Improving Generalization in LLM Structured Pruning via Function-Aware Neuron Groupingcited by 2
- 2026GRASPrune: Global Gating for Budgeted Structured Pruning of Large Language Modelscited by 2
- 2025StructPrune: Structured Global Pruning asymptotics with O(√N) GPU Memorycited by 1
- 2026GPrune-LLM: Generalization-Aware Structured Pruning for Large Language Modelscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026OS-Pruner: Pruning Chains-of-Thought of Reasoning Models via Optimal Stopping
- 2025Can Pruning Improve Reasoning? Revisiting Long-CoT Compression with Capability in Mind for Better Reasoning
- 2025Fractured Chain-of-Thought Reasoning
- 2025Reasoning Models Can be Accurately Pruned Via Chain-of-Thought Reconstruction
- 2025R-Capsule: Compressing High-Level Plans for Efficient Large Language Model Reasoning
- 2026Latent Reasoning with Normalizing Flows
- 2025Inference-Time Chain-of-Thought Pruning with Latent Informativeness Signals
- 2025When Fewer Layers Break More Chains: Layer Pruning Harms Test-Time Scaling in LLMs