Training LLMs for Divide-and-Conquer Reasoning Elevates Test-Time Scalability
On the map: LLM reasoning → Reinforcement learning for reasoning
Abstract (arXiv)
Large language models (LLMs) have demonstrated strong reasoning capabilities through step-by-step chain-of-thought (CoT) reasoning. Nevertheless, at the limits of model capability, CoT often proves insufficient, and its strictly sequential nature constrains test-time scalability. A potential alternative is divide-and-conquer (DAC) reasoning, which decomposes a complex problem into subproblems to facilitate more effective exploration of the solution. Although promising, our analysis reveals a fundamental misalignment between general-purpose post-training and DAC-style inference, which limits the model's capacity to fully leverage this potential. To bridge this gap and fully unlock LLMs' reasoning capabilities on the most challenging tasks, we propose an end-to-end reinforcement learning (RL) framework to enhance their DAC-style reasoning capacity. At each step, the policy decomposes a problem into a group of subproblems, solves them sequentially, and addresses the original one conditioned on the subproblem solutions, with both decomposition and solution integrated into RL training. Under comparable training, our DAC-style framework endows the model with a higher performance ceiling and stronger test-time scalability, surpassing CoT by 8.6% in Pass@1 and 6.3% in Pass@32 on competition-level benchmarks.
Key facts. Cited by 4 papers in the map (data through 13 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 43 of its 56 references that are in the map. First posted on arXiv 2 Feb 2026.
Builds on
43 of 56 references.
Closest ancestors · 10
Same direction, newest first
- 2025rStar2-Agent: Agentic Reasoning Technical Report28 Aug
- 2025Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVR19 Aug
- 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs17 Jun
- 2025SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoning10 Jun
- 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Example29 Apr
- 2025VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks7 Apr
- 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
- 2025Process Reinforcement through Implicit Rewards3 Feb
- 2017Proximal Policy Optimization Algorithms20 Jul
Other references · 33
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,376
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,091
- 2021Evaluating Large Language Models Trained on Codecited by 8,205
- 2020Measuring Massive Multitask Language Understandingcited by 7,876
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,798
- 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,053
- 2021Program Synthesis with Large Language Modelscited by 3,573
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,181
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,804
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,951
- 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,785
- 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,404
21 more
- 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,077
- 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,066
- 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,032
- 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 879
- 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 556
- 2025Demystifying Long Chain-of-Thought Reasoning in LLMscited by 325
- 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
- 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 281
- 2025DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decompositioncited by 280
- 2024ReFT: Reasoning with Reinforced Fine-Tuningcited by 237
- 2023Mitigating the Alignment Tax of RLHFcited by 183
- 2023Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoningcited by 178
- 2023Deductive Verification of Chain-of-Thought Reasoningcited by 123
- 2025Seed-Prover: Deep and Broad Reasoning for Automated Theorem Provingcited by 97
- 2022Successive Prompting for Decomposing Complex Questionscited by 94
- 2024Buffer of Thoughts: Thought-Augmented Reasoning with Large Language Modelscited by 91
- 2022Parsel: Algorithmic Reasoning with Language Models by Composing Decompositionscited by 63
- 2025Chain-of-Reasoning: Towards Unified Mathematical Reasoning in Large Language Models via a Multi-Paradigm Perspectivecited by 29
- 2025LADDER: Self-Improving LLMs Through Recursive Problem Decompositioncited by 24
- 2025Dynamic Generation of Multi-LLM Agents Communication Topologies with Graph Diffusion Modelscited by 11
- 2025Mitigating Over-Refusal in Aligned Large Language Models via Inference-Time Activation Energycited by 4
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026Beyond Outliers: A Data-Free Layer-wise Mixed-Precision Quantization Approach Driven by Numerical and Structural Dual-Sensitivitycited by 5
- 2026Bridging the Detection-to-Abstention Gap in Reasoning Models under Insufficient Informationcited by 2
- 2026From Solvers to Research: Large Language Model-Driven Formal Mathematics at the Research Frontiercited by 1
- 2026Efficient Reasoning Exploration via State-Conditioned Latent Steering with Progress Guidancecited by 0
- 2026Efficient Reasoning Exploration via State-Conditioned Latent Steering with Progress Guidance21 Sep 2026
- 2026From Solvers to Research: Large Language Model-Driven Formal Mathematics at the Research Frontier8 Jul 2026
- 2026Bridging the Detection-to-Abstention Gap in Reasoning Models under Insufficient Information27 May 2026
- 2026Beyond Outliers: A Data-Free Layer-wise Mixed-Precision Quantization Approach Driven by Numerical and Structural Dual-Sensitivity18 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2025Interleaved Reasoning for Large Language Models via Reinforcement Learning
- 2026Re2: Unlocking LLM Reasoning via Reinforcement Learning with Re-solving
- 2026The Art of Efficient Reasoning: Data, Reward, and Optimization
- 2025Divide and Conquer: Grounding LLMs as Efficient Decision-Making Agents via Offline Hierarchical Reinforcement Learning
- 2025Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
- 2025Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search
- 2026Policy of Thoughts: Scaling Test-Time Training for LLM Reasoning via Online Policy Evolution
- 2026Correct, Concise and Complete: Multi-stage Training For Adaptive Reasoning