# Revisiting Model Interpolation for Efficient Reasoning

arXiv: https://arxiv.org/abs/2510.10977 · HTML: https://mlsanity.com/paper/2510.10977/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 15 of its 27 references that are in the map.
- First posted on arXiv 13 Oct 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 15 of 15)

- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch](https://mlsanity.com/paper/2311.03099/) (2023)
- [Chain of Draft: Thinking Faster by Writing Less](https://mlsanity.com/paper/2502.18600/) (2025)
- [Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities](https://mlsanity.com/paper/2408.07666/) (2024)
- [AdaptThink: Reasoning Models Can Learn When to Think](https://mlsanity.com/paper/2505.13417/) (2025)
- [Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketching](https://mlsanity.com/paper/2503.05179/) (2025)
- [Fast Best-of-N Decoding via Speculative Rejection](https://mlsanity.com/paper/2410.20290/) (2024)
- [Soft Merging of Experts with Adaptive Routing](https://mlsanity.com/paper/2306.03745/) (2023)
- [Sampling-Efficient Test-Time Scaling: Self-Estimating the Best-of-N Sampling in Early Decoding](https://mlsanity.com/paper/2503.01422/) (2025)
- [Unlocking Efficient Long-to-Short LLM Reasoning with Model Merging](https://mlsanity.com/paper/2503.20641/) (2025)
- [Z1: Efficient Test-time Scaling with Code](https://mlsanity.com/paper/2504.00810/) (2025)
- [Towards Efficient Pareto Set Approximation via Mixture of Experts Based Model Fusion](https://mlsanity.com/paper/2406.09770/) (2024)
- [Timber: Training-free Instruct Model Refining with Base via Effective Rank](https://mlsanity.com/paper/2509.23595/) (2025)
- [Merge Now, Regret Later: The Hidden Cost of Model Merging Is Adversarial Transferability](https://mlsanity.com/paper/2509.23689/) (2025)

## Built on by (most cited first, 10 of 10)

- [OckBench: Measuring the Efficiency of LLM Reasoning](https://mlsanity.com/paper/2511.05722/) (2025)
- [Behavior Knowledge Merge in Reinforced Agentic Models](https://mlsanity.com/paper/2601.13572/) (2026)
- [The Art of Efficient Reasoning: Data, Reward, and Optimization](https://mlsanity.com/paper/2602.20945/) (2026)
- [QA-Merging: Query-Adaptive Reasoning via Layer Selective Model Merging](https://mlsanity.com/paper/2601.03506/) (2026)
- [AP-BMM: Approximating Capability-Cost Pareto Sets of LLMs via Asynchronous Prior-Guided Bayesian Model Merging](https://mlsanity.com/paper/2512.09972/) (2025)
- [Can We Trust LLMs on Memristors? Diving into Reasoning Ability under Non-Ideality](https://mlsanity.com/paper/2603.13725/) (2026)
- [Graph-Based Chain-of-Thought Pruning for Reducing Redundant Reflections in Reasoning LLMs](https://mlsanity.com/paper/2604.05643/) (2026)
- [GRIP: Granular Reward-Guided Parameter Interpolation for Efficient Reasoning](https://mlsanity.com/paper/2608.25583/) (2026)
- [MI-Distillation: Selecting from Model-Interpolated Instruct-Reasoning Data Spectrum for Chain-of-Thought Distillation](https://mlsanity.com/paper/2608.29623/) (2026)
- [Lightning Weave: Improving the Accuracy-Efficiency Frontier of Reasoning Models through Capability Composition](https://mlsanity.com/paper/2609.14708/) (2026)

## Abstract (arXiv)

Model merging, typically on Instruct and Thinking models, has shown remarkable performance for efficient reasoning. In this paper, we systematically revisit the simplest merging method that interpolates two weights directly. Particularly, we observe that model interpolation follows a three-stage evolutionary paradigm with distinct behaviors on the reasoning trajectory. These dynamics provide a principled guide for navigating the performance-cost trade-off. Empirical results demonstrate that a strategically interpolated model surprisingly surpasses sophisticated model merging baselines on both efficiency and effectiveness. We further validate our findings with extensive ablation studies on model layers, modules, and decoding strategies. Ultimately, this work demystifies model interpolation and offers a practical framework for crafting models with precisely targeted reasoning capabilities. Code is available at hrefhttps://github.com/wutaiqiang/MIGithub.