mlsanityData to 14 Oct 2026

The Thinking Spectrum: An Empirical Study of Tunable Reasoning in LLMs through Model Merging

arXiv 2509.22034 ↗First posted 26 Sep 2025cs.AI cs.CLcited by 4

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

The growing demand for large language models (LLMs) with tunable reasoning capabilities in many real-world applications highlights a critical need for methods that can efficiently produce a spectrum of models balancing reasoning depth and computational cost. Model merging has emerged as a promising, training-free technique to address this challenge by arithmetically combining the weights of a general-purpose model with a specialized reasoning model. While various merging techniques exist, their potential to create a spectrum of models with fine-grained control over reasoning abilities remains largely unexplored. This work presents a large-scale empirical study evaluating a range of model merging techniques across multiple reasoning benchmarks. We systematically vary merging strengths to construct accuracy-efficiency curves, providing the first comprehensive view of the tunable performance landscape. Our findings reveal that model merging offers an effective and controllable method for calibrating the trade-off between reasoning accuracy and token efficiency, even when parent models have highly divergent weight spaces. Crucially, we identify instances of Pareto Improvement, where a merged model achieves both higher accuracy and lower token consumption than one of its parents. Our study provides the first comprehensive analysis of this tunable space, offering practical guidelines for creating LLMs with specific reasoning profiles to meet diverse application demands.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 29 of its 48 references that are in the map. First posted on arXiv 26 Sep 2025.

Builds on

29 of 48 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025Plan and Budget: Effective and Efficient Test-Time Scaling on Reasoning Large Language Models22 May
  2. 2025Efficient Reasoning Models: A Survey15 Apr
  3. 2025Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models7 Apr
  4. 2025When Reasoning Meets Compression: Understanding the Effects of LLMs Compression on Large Reasoning Models2 Apr
  5. 2025ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning2 Apr
  6. 2025Unlocking Efficient Long-to-Short LLM Reasoning with Model Merging26 Mar
  7. 2025Sampling-Efficient Test-Time Scaling: Self-Estimating the Best-of-N Sampling in Early Decoding3 Mar
  8. 2025Self-Training Elicits Concise Reasoning in Large Language Models27 Feb
  9. 2025Reasoning with Latent Thoughts: On the Power of Looped Transformers24 Feb
  10. 2025Dynamic Parallel Tree Search for Efficient LLM Reasoning22 Feb
  11. 2025TokenSkip: Controllable Chain-of-Thought Compression in LLMs17 Feb
  12. 2025Efficient Reasoning with Hidden Thinking31 Jan
1 more
  1. 2025O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning22 Jan

Other references · 16

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 2024GPT-4o System Cardcited by 4,588
  4. 2022Emergent Abilities of Large Language Modelscited by 2,512
  5. 2024OpenAI o1 System Cardcited by 1,244
  6. 2024DeepSeek-V3 Technical Reportcited by 811
  7. 2025Kimi K2: Open Agentic Intelligencecited by 567
  8. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 511
  9. 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 471
  10. 2024Arcee's MergeKit: A Toolkit for Merging Large Language Modelscited by 194
  11. 2024DELLA-Merging: Reducing Interference in Model Merging through Magnitude-Based Samplingcited by 81
  12. 2024FuseChat: Knowledge Fusion of Chat Modelscited by 55
4 more
  1. 2024Predicting Emergent Capabilities by Finetuningcited by 10
  2. 2025LoRE-Merging: Exploring Low-Rank Estimation For Large Language Model Mergingcited by 7
  3. 2024Plato: Plan to Efficiently Decode for Large Language Model Inferencecited by 6
  4. 2024textitSKIntern: Internalizing Symbolic Knowledge for Distilling Better CoT Capabilities into Small Language Modelscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024Unconstrained Model Merging for Enhanced LLM Reasoning
  2. 2025Fine, I'll Merge It Myself: A Multi-Fidelity Framework for Automated Model Merging
  3. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMs
  4. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
  5. 2025A Survey of Scaling in Large Language Model Reasoning
  6. 2024Inference Scaling vs Reasoning: An Empirical Analysis of Compute-Optimal LLM Problem-Solving
  7. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Models
  8. 2026Tandem: Riding Together with Large and Small Language Models for Efficient Reasoning