mlsanityData to 7 Oct 2026

Fusion Training for Mathematical Generalization in Large Language Models

arXiv 2608.09893 ↗First posted 10 Aug 2026cs.CL cs.AIcited by 0

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

Thinking Mode Fusion (TMF) enables large language models to support both concise responses and long-form reasoning by unifying a non-thinking mode and a thinking mode within a single model. However, its training dynamics, including the data ratio and training schedule between the two modes, remain underexplored. In this work, we present a systematic study of TMF by analyzing the effects of the training schedule and data ratio between thinking and non-thinking modes. Focusing on mathematical problem solving, we construct a benchmark with multiple thinking-to-non-thinking data ratios and three training schedules. Our results reveal an asymmetric interaction between the two modes: increasing the ratio of non-thinking supervision reduces the accuracy of the thinking mode. We further show that different training schedules modulate this trade-off and that the optimal schedule depends on the data ratio. Finally, we quantify a negative correlation between non-thinking and thinking mode supervision, highlighting an inherent tension between these two modes. These findings provide practical guidance for designing effective TMF training settings. All code and data are released to support further research at: hrefhttps://github.com/caocongfeng/Fusion-Bench.gitFusion Bench.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 20 of its 28 references that are in the map. First posted on arXiv 10 Aug 2026.

Builds on

20 of 28 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning dataset23 Apr
  2. 2025LIMO: Less is More for Reasoning5 Feb
  3. 2024OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data2 Oct
  4. 2024Skywork-Math: Data Scaling Laws for Mathematical Reasoning in Large Language Models -- The Story Goes On11 Jul
  5. 2024OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset15 Feb
  6. 2023Calc-X and Calcformers: Empowering Arithmetical Chain-of-Thought through Interaction with Symbolic Systems24 May

Other references · 14

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  3. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  4. 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoningcited by 257
  5. 2025Training Language Models to Reason Efficientlycited by 242
  6. 2024Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performancecited by 179
  7. 2024RegMix: Data Mixture as Regression for Language Model Pre-trainingcited by 172
  8. 2024The Impact of Reasoning Step Length on Large Language Modelscited by 152
  9. 2025AdaptThink: Reasoning Models Can Learn When to Thinkcited by 141
  10. 2024Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuningcited by 85
  11. 2025Unlocking Efficient Long-to-Short LLM Reasoning with Model Mergingcited by 58
  12. 2023LongForm: Effective Instruction Tuning with Reverse Instructionscited by 47
2 more
  1. 2025Measuring Chain of Thought Faithfulness by Unlearning Reasoning Stepscited by 43
  2. 2025Demystifying Hybrid Thinking: Can LLMs Truly Switch Between Think and No-Think?cited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey on Large Language Models for Mathematical Reasoning
  2. 2025Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoning
  3. 2025Teaching LLMs to Think Mathematically: A Critical Study of Decision-Making via Optimization
  4. 2023Specializing Smaller Language Models towards Multi-Step Reasoning
  5. 2025DynamicMind: A Tri-Mode Thinking System for Large Language Models
  6. 2025How and Why LLMs Generalize: A Fine-Grained Analysis of LLM Reasoning from Cognitive Behaviors to Low-Level Patterns
  7. 2024MindStar: Enhancing Math Reasoning in Pre-trained LLMs at Inference Time
  8. 2026Thinking vs. NoThinking: Towards Interpreting Reasoning Mechanisms of Large Language Models via Sparse Autoencoders