mlsanityData to 7 Oct 2026

Weight Ensembling Improves Reasoning in Language Models

arXiv 2504.10478 ↗First posted 14 Apr 2025cs.LG cs.AIcited by 36

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

We investigate a failure mode that arises during the training of reasoning models, where the diversity of generations begins to collapse, leading to suboptimal test-time scaling. Notably, the Pass@1 rate reliably improves during supervised finetuning (SFT), but Pass@k rapidly deteriorates. Surprisingly, a simple intervention of interpolating the weights of the latest SFT checkpoint with an early checkpoint, otherwise known as WiSE-FT, almost completely recovers Pass@k while also improving Pass@1. The WiSE-FT variant achieves better test-time scaling (Best@k, majority vote) and achieves superior results with less data when tuned further by reinforcement learning. Finally, we find that WiSE-FT provides complementary performance gains that cannot be achieved only through diversity-inducing decoding strategies, like temperature scaling. We formalize a bias-variance tradeoff of Pass@k with respect to the expectation and variance of Pass@1 over the test distribution. We find that WiSE-FT can reduce bias and variance simultaneously, while temperature scaling inherently trades off between bias and variance.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 28 of its 35 references that are in the map. First posted on arXiv 14 Apr 2025.

Builds on

28 of 35 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Rethinking Fine-Tuning when Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoning11 Feb
  2. 2025Demystifying Long Chain-of-Thought Reasoning in LLMs5 Feb
  3. 2025s1: Simple test-time scaling31 Jan
  4. 2024Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Models18 Dec
  5. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
  6. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models1 Aug

Other references · 22

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2023Let's Verify Step by Stepcited by 3,598
  6. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  7. 2021Robust fine-tuning of zero-shot modelscited by 785
  8. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
  9. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 655
  10. 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 299
  11. 2024Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoningcited by 267
  12. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 231
10 more
  1. 2023Is Self-Repair a Silver Bullet for Code Generation?cited by 191
  2. 2024AlphaMath Almost Zero: Process Supervision without Processcited by 160
  3. 2024Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputscited by 148
  4. 2024Jasper and Stella: distillation of SOTA embedding modelscited by 109
  5. 2024Self-Improvement in Language Models: The Sharpening Mechanismcited by 102
  6. 2024BOND: Aligning LLMs with Best-of-N Distillationcited by 73
  7. 2024Forcing Diffuse Distributions out of Language Modelscited by 48
  8. 2017Generating High-Quality and Informative Conversation Responses with Sequence-to-Sequence Modelscited by 42
  9. 2022Submodularity In Machine Learning and Artificial Intelligencecited by 37
  10. 2024Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Modelscited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 17; 2026: 19.

36 papers.

Sort:

By area:

  1. 2025The Surprising Effectiveness of Negative Reinforcement in LLM Reasoningcited by 158
  2. 2025Outcome-based Exploration for LLM Reasoningcited by 79
  3. 2025Rewarding the Unlikely: Lifting GRPO Beyond Distribution Sharpeningcited by 78
  4. 2025RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMscited by 32
  5. 2025KL-Regularized Reinforcement Learning is Designed to Mode Collapsecited by 29
  6. 2025Roll the dice & look before you leap: Going beyond the creative limits of next-token predictioncited by 28
  7. 2025Differential Smoothing Mitigates Sharpening and Improves LLM Reasoningcited by 19
  8. 2025Representation-Based Exploration for Language Models: From Test-Time to Post-Trainingcited by 16
  9. 2025The Debate on RLVR Reasoning Capability Boundary: Shrinkage, Expansion, or Both? A Two-Stage Dynamic Viewcited by 13
  10. 2026Replaying pre-training data improves fine-tuningcited by 12
  11. 2025Unveiling Over-Memorization in Finetuning LLMs for Reasoning Taskscited by 10
  12. 2025Reasoning Path Divergence: A New Metric and Curation Strategy to Unlock LLM Diverse Thinkingcited by 10
24 more
  1. 2025Understanding and Steering the Cognitive Behaviors of Reasoning Models at Test-Timecited by 10
  2. 2026F-GRPO: Don't Let Your Policy Learn the Obvious and Forget the Rarecited by 10
  3. 2025Diversity-Enhanced Reasoning for Subjective Questionscited by 9
  4. 2025Revisiting Test-Time Scaling: A Survey and a Diversity-Aware Method for Efficient Reasoningcited by 8
  5. 2025Distilled Pretraining: A modern lens of Data, In-Context Learning and Test-Time Scalingcited by 7
  6. 2026LaDi-RL: Latent Diffusion Reasoning Prevents Entropy Collapse in Reinforcement Learningcited by 7
  7. 2026Steer2Adapt: Dynamically Composing Steering Vectors Elicits Efficient Adaptation of LLMscited by 7
  8. 2025How Does RL Post-training Induce Skill Composition? A Case Study on Countdowncited by 6
  9. 2026Do Instruction-Tuned Models Always Perform Better Than Base Models? Evidence from Math and Domain-Shifted Benchmarkscited by 4
  10. 2026Teacher-Free Self-Training Amplifies but Does Not Compound: A Pass@K Crossover on a Free-Verifier Domaincited by 4
  11. 2025When Inverse Data Outperforms: Exploring the Pitfalls of Mixed Data in Multi-Stage Fine-Tuningcited by 2
  12. 2026Exploring Diverse Generation Paths via Inference-time Stiefel Activation Steeringcited by 2
  13. 2026On Advantage Estimates for Max@K Policy Gradientscited by 2
  14. 2026Beyond the Best Guess: Improving LLM Solution Coverage with Evolution Strategiescited by 2
  15. 2026Computational Arbitrage in AI Model Marketscited by 1
  16. 2026OrderGrad: Optimizing Beyond the Mean with Order-Statistic Policy Gradient Estimationcited by 1
  17. 2026Learning Mechanistic Reasoning for Chemical Reactions with Large Language Modelscited by 1
  18. 2026Rate or Fate? RLV^varepsilonR: Reinforcement Learning with Verifiable Noisy Rewardscited by 0
  19. 2026AIPO: Learning to Reason from Active Interactioncited by 0
  20. 2026Reasoning Portability: Guiding Continual Learning for MLLMs in the RLVR Eracited by 0
  21. 2026Fork-Think with Confidencecited by 0
  22. 2026When Does the Best Sampling Temperature Rise with the Budget? Sufficient Conditions for Pass@kcited by 0
  23. 2026Fine Until Fine-Tuned: Repeated Solutions Make Reasoning Fragilecited by 0
  24. 2026Adapter Thickets: Splitting an RLVR Budget Beats Concentrating Itcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025On the Role of Temperature Sampling in Test-Time Scaling
  2. 2025Less is More: Improving LLM Reasoning with Minimal Test-Time Intervention
  3. 2025Rethinking Inference-Time Scaling: Efficiency Limits and Linguistic Signals
  4. 2025Enhancing Large Language Model Reasoning via Selective Critical Token Fine-Tuning
  5. 2025Making Small Language Models Efficient Reasoners: Intervention, Supervision, Reinforcement
  6. 2025Step-level Verifier-guided Hybrid Test-Time Scaling for Large Language Models
  7. 2026Bridging SFT and RL: Dynamic Policy Optimization for Robust Reasoning
  8. 2026Stabilizing Efficient Reasoning with Step-Level Advantage Selection