mlsanityData to 7 Oct 2026

Relative Kinetic Utility for Reasoning-Aware Structural Pruning in Large Language Models

arXiv 2605.09008 ↗First posted 9 May 2026cs.LG cs.CLcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Global structured pruning requires channels from different layers to compete under a shared sparsity budget, raising two coupled challenges: identifying which channels should be retained and making their scores comparable across layers. Raw channel scores can contain block-common scale that leaves within-block ordering unchanged but distorts model-wide competition. Our experiment indicates that similar layer-wise allocations can retain substantially different FFN channels, so layer allocation alone does not determine channel identity. Motivated by this separation, we introduce Global Relative Kinetic Utility (Global RKU), a label-free criterion that separates channel importance estimation from cross-layer comparison. Global RKU measures channel participation using a final-hidden-state activation-gradient signal, then applies block-relative normalization to mitigate block-common scale while preserving within-block ordering, requires only unlabeled calibration inputs, and produces a static pruning topology in a single calibration stage. Under questions-only calibration on Qwen-2.5-7B, RKU-GISP Mean3 margins are -0.98, +3.79, and +8.61 points at 30%, 40%, and 50% sparsity, respectively (average +3.81). Additional Qwen evaluations cover non-mathematical reasoning, recovery, held-out transfer, and physical deployment. Separately, replacing Wiki16K with questions-only Q16K improves RKU's Mean3 at every tested sparsity on Qwen, Llama, and Gemma. Our ablation study shows relative-normalization gains of 14.42 and 5.53 Mean3 points at 40% and 50% sparsity, respectively; the common-seed audit is positive in all 27 seed-task comparisons.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 42 of its 46 references that are in the map. First posted on arXiv 9 May 2026.

Builds on

42 of 46 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026From LLMs to LRMs: Rethinking Pruning for Reasoning-Centric Models26 Jan
  2. 2025Think Before You Prune: Self-Reflective Structured Pruning for Reasoning Language Models1 Dec
  3. 2025Think Before You Prune: Selective Self-Generated Calibration for Pruning Large Reasoning Models24 Nov
  4. 2024Rethinking Pruning Large Language Models: Benefits and Pitfalls of Reconstruction Error Minimization21 Jun

Other references · 38

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  3. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 844
  4. 2019Are Sixteen Heads Really Better than One?cited by 803
  5. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 802
  6. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
  7. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  8. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 643
  9. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  10. 2019Importance Estimation for Neural Network Pruningcited by 531
  11. 2023Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruningcited by 408
  12. 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 378
26 more
  1. 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expectcited by 369
  2. 2021Block Pruning For Faster Transformerscited by 176
  3. 2022Structured Pruning Learns Compact and Accurate Modelscited by 167
  4. 2023Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsitycited by 163
  5. 2023Fluctuation-based Adaptive Structured Pruning for Large Language Modelscited by 110
  6. 2023LoRAPrune: Structured Pruning Meets Low-Rank Parameter-Efficient Fine-Tuningcited by 53
  7. 2023ZipLM: Inference-Aware Structured Pruning of Language Modelscited by 46
  8. 2024BESA: Pruning Large Language Models with Blockwise Parameter-Efficient Sparsity Allocationcited by 44
  9. 2024DISP-LLM: Dimension-Independent Structural Pruning for Large Language Modelscited by 24
  10. 2024SparseLLM: Towards Global Pruning for Pre-trained Language Modelscited by 23
  11. 2024Is C4 Dataset Optimal for Pruning? An Investigation of Calibration Data for LLM Pruningcited by 19
  12. 2025SlimLLM: Accurate Structured Pruning for Large Language Modelscited by 18
  13. 2025DarwinLM: Evolutionary Structured Pruning of Large Language Modelscited by 16
  14. 2024Beware of Calibration Data for Pruning Large Language Modelscited by 14
  15. 2025Instruction-Following Pruning for Large Language Modelscited by 14
  16. 2025Probe Pruning: Accelerating LLMs through Dynamic Pruning via Model-Probingcited by 14
  17. 2025Alternating Gradient Flows: A Theory of Feature Learning in Two-layer Neural Networkscited by 11
  18. 2024On the Limitations of Language Targeted Pruning: Investigating the Calibration Language Impact in Multilingual LLM Pruningcited by 6
  19. 2025Olica: Efficient Structured Pruning of Large Language Models without Retrainingcited by 6
  20. 2025NIRVANA: Structured Pruning Reimagined for Large Language Model Compressioncited by 6
  21. 2025From Local to Global: Revisiting Structured Pruning Paradigms for Large Language Modelscited by 5
  22. 2025RCPU: Rotation-Constrained Error Compensation for Structured Pruning of Large Language Modelscited by 2
  23. 2025Improving Generalization in LLM Structured Pruning via Function-Aware Neuron Groupingcited by 2
  24. 2026GRASPrune: Global Gating for Budgeted Structured Pruning of Large Language Modelscited by 2
  25. 2025StructPrune: Structured Global Pruning asymptotics with O(√N) GPU Memorycited by 1
  26. 2026GPrune-LLM: Generalization-Aware Structured Pruning for Large Language Modelscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026OS-Pruner: Pruning Chains-of-Thought of Reasoning Models via Optimal Stopping
  2. 2025Can Pruning Improve Reasoning? Revisiting Long-CoT Compression with Capability in Mind for Better Reasoning
  3. 2025Fractured Chain-of-Thought Reasoning
  4. 2025Reasoning Models Can be Accurately Pruned Via Chain-of-Thought Reconstruction
  5. 2025R-Capsule: Compressing High-Level Plans for Efficient Large Language Model Reasoning
  6. 2026Latent Reasoning with Normalizing Flows
  7. 2025Inference-Time Chain-of-Thought Pruning with Latent Informativeness Signals
  8. 2025When Fewer Layers Break More Chains: Layer Pruning Harms Test-Time Scaling in LLMs