# Revisiting the Effectiveness of LLM Pruning for Test-Time Scaling

arXiv: https://arxiv.org/abs/2604.25098 · HTML: https://mlsanity.com/paper/2604.25098/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 11 of its 46 references that are in the map.
- First posted on arXiv 28 Apr 2026.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 11 of 11)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [SpinQuant: LLM quantization with learned rotations](https://mlsanity.com/paper/2405.16406/) (2024)
- [Beyond Efficiency: A Systematic Survey of Resource-Efficient Large Language Models](https://mlsanity.com/paper/2401.00625/) (2024)
- [Interpretable Contrastive Monte Carlo Tree Search Reasoning](https://mlsanity.com/paper/2410.01707/) (2024)
- [Pack of LLMs: Model Fusion at Test-Time via Perplexity Optimization](https://mlsanity.com/paper/2404.11531/) (2024)
- [When Fewer Layers Break More Chains: Layer Pruning Harms Test-Time Scaling in LLMs](https://mlsanity.com/paper/2510.22228/) (2025)
- [Less Diverse, Less Safe: The Indirect But Pervasive Risk of Test-Time Scaling in Large Language Models](https://mlsanity.com/paper/2510.08592/) (2025)
- [High-Fidelity Pruning for Large Language Models](https://mlsanity.com/paper/2603.08083/) (2026)

## Built on by (most cited first, 2 of 2)

- [Weight-Adjusted Gradients Reveal Parameter Importance and Failure Modes in LLMs](https://mlsanity.com/paper/2607.10803/) (2026)
- [ReCal: Calibrating Structured Pruning for On-Policy Distillation Recovery](https://mlsanity.com/paper/2610.11332/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) now exhibit remarkable reasoning capabilities through test-time compute scaling (TTS), with impressive performance across math and coding benchmarks. In parallel, research in model compression has developed pruning methods that seek to remove redundant/detrimental parameters without sacrificing task performance. The intersection of these two research advancements lays the foundation for our work. Specific to reasoning LLMs, prior work has shown that structured pruning (methods which remove entire set of layer blocks), significantly degrades TTS reasoning performance. However, in this work, we revisit this assumption and investigate whether unstructured pruning (methods that carefully remove only certain redundant/detrimental weights) exhibits similar limitations. Surprisingly, our extensive experiments across four reasoning benchmarks on two reasoning LLMs: s1.1-7B and Qwen3-8B, consistently show that unstructured pruning augments TTS performance compared to structured pruning, and at times can even outperform the unpruned full-weight LLMs. Furthermore, we also empirically study the impact of different layer-wise sparsity allocation strategies, which are an important parametric choice for instantiating these unstructured methods. These findings challenge the conventional notion that pruning always reduces TTS performance and in fact, suggest that carefully undertaken pruning can retain TTS effectiveness.