mlsanityData to 14 Oct 2026

Scalable LLM Reasoning Acceleration with Low-rank Distillation

arXiv 2505.07861 ↗First posted 8 May 2025cs.CL cs.AI cs.LGcited by 7

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Due to long generations, large language model (LLM) math reasoning demands significant computational resources and time. While many existing efficient inference methods have been developed with excellent performance preservation on language tasks, they often severely degrade math performance. In this paper, we propose Caprese, a resource-efficient distillation method to recover lost capabilities from deploying efficient inference methods, focused primarily in feedforward blocks. With original weights unperturbed, roughly 1% of additional parameters, and only 20K synthetic training samples, we are able to recover much if not all of the reasoning capabilities lost from efficient inference for thinking LLMs and without harm to language tasks for instruct LLMs. Moreover, Caprese slashes the number of active parameters (~2B cut for Gemma 2 9B and Llama 3.1 8B) and integrates cleanly into existing model layers to reduce latency (>16% time-to-next-token reduction) while encouraging response brevity (up to 8.5% fewer tokens).

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 46 of its 55 references that are in the map. First posted on arXiv 8 May 2025.

Builds on

46 of 55 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025s1: Simple test-time scaling31 Jan
  2. 2024Adaptive Inference-Time Compute: LLMs Can Predict if They Can Do Better, Even Mid-Generation3 Oct
  3. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
  4. 2024Concise Thoughts: Impact of Output Length on LLM Reasoning and Cost29 Jul

Other references · 42

  1. 2017Attention Is All You Needcited by 60,361
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2020Scaling Laws for Neural Language Modelscited by 6,586
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  7. 2024Qwen2.5 Technical Reportcited by 4,942
  8. 2023Let's Verify Step by Stepcited by 3,789
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  10. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  11. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  12. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
30 more
  1. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  2. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  3. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  4. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
  5. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  6. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,292
  7. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  8. 2015Teaching Machines to Read and Comprehendcited by 1,049
  9. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
  10. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  11. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  12. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 867
  13. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  14. 2018CoQA: A Conversational Question Answering Challengecited by 683
  15. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 650
  16. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 486
  17. 2025Chain of Draft: Thinking Faster by Writing Lesscited by 297
  18. 2023Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Timecited by 252
  19. 2025Training Language Models to Reason Efficientlycited by 247
  20. 2021MoEfication: Transformer Feed-forward Layers are Mixtures of Expertscited by 152
  21. 2025Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuningcited by 123
  22. 2022The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformerscited by 122
  23. 2025Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketchingcited by 117
  24. 2024Fast Best-of-N Decoding via Speculative Rejectioncited by 112
  25. 2024Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inferencecited by 75
  26. 2024The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Modelscited by 63
  27. 2024CATS: Contextually-Aware Thresholding for Sparsity in Large Language Modelscited by 41
  28. 2024Prompt-prompted Adaptive Structured Pruning for Efficient LLM Generationcited by 21
  29. 2024Towards Low-bit Communication for Tensor Parallel LLM Inferencecited by 7
  30. 2024Sirius: Contextual Sparsity with Correction for Efficient LLMscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 3.

7 papers.

  1. 2025Population-Aligned Persona Generation for LLM-based Social Simulationcited by 21
  2. 2025Generalized Parallel Scaling with Interdependent Generationscited by 6
  3. 2026Learning Adaptive Reasoning Paths for Efficient Visual Reasoningcited by 5
  4. 2025PISA-Bench: The PISA Index as a Multilingual and Multimodal Metric for the Evaluation of Vision-Language Modelscited by 3
  5. 2026Prefix Sliding for efficient test-time scalingcited by 2
  6. 2025Systematic Optimization of Open Source Large Language Models for Mathematical Reasoningcited by 0
  7. 2026Temporal Stability and Few-Shot Prompting in Math Task Assessmentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Improving Mathematical Reasoning Capabilities of Small Language Models via Feedback-Driven Distillation
  2. 2024MindStar: Enhancing Math Reasoning in Pre-trained LLMs at Inference Time
  3. 2026FABSVer: Faster Training and Better Self-Verification for LLM Mathematical Reasoning
  4. 2024Improving LLM Reasoning through Scaling Inference Computation with Collaborative Verification
  5. 2025DeepDistill: Enhancing LLM Reasoning Capabilities via Large-Scale Difficulty-Graded Data Training
  6. 2025Do LLMs Overthink Basic Math Reasoning? Benchmarking the Accuracy-Efficiency Tradeoff in Language Models
  7. 2025SABER: Switchable and Balanced Training for Efficient LLM Reasoning
  8. 2025xLSTM 7B: A Recurrent LLM for Fast and Efficient Inference