mlsanityData to 14 Oct 2026

Expanding LLM Reasoning

arXiv 2610.05584 ↗First posted 4 Oct 2026cs.LG cs.AI cs.CLcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Extra inference compute is usually spent on sampling more reasoning chains. We study where inside an existing chain an additional continuation should begin. We define expansion utility, the change in correctness from restarting a chain at a stored step, and measure it at every eligible step for nine models on six benchmarks (41 model and benchmark cells). Restart position matters: steps selected on one set of continuations beat uniform placement when scored on disjoint ones, in held-out audits on 5, 16, and 38 cells (+4.25 points [+2.51, +6.63] in a fresh five-cell audit). A fixed rule that restarts from the last eligible steps, always-last, is a strong baseline: our learned router beats uniform placement but shows no detected gain over it, and on DeepSeek-R1-Distill-Qwen-14B/MATH-500 always-last exceeds the exact self-consistency frontier at matched aggregate generated output by +0.052 [+0.008, +0.098], using 0.774x the aggregate generated output of four-sample self-consistency. Cross-fitted oracle selection still finds held-out headroom beyond declared positional classes, a target for future selectors. Finally, breaking step-label ties by earliest index flips the sign of a pointwise selector's gain over uniform placement in every seed of a five-seed diagnostic with four rollouts per step; randomized ties remove the bias.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 12 of its 29 references that are in the map. First posted on arXiv 4 Oct 2026.

Builds on

12 of 29 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug

Other references · 11

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2021Program Synthesis with Large Language Modelscited by 3,637
  5. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  6. 2025Gemma 3 Technical Reportcited by 1,824
  7. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  8. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  9. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 506
  10. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 441
  11. 2025Thought Anchors: Which LLM Reasoning Steps Matter?cited by 129

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026It's the Problem, Not the Path: Budget and Difficulty Confounds in LLM Reasoning Trajectories
  2. 2025Meaningless Tokens, Meaningful Gains: How Activation Shifts Enhance LLM Reasoning
  3. 2026ReFlect: An Effective Harness System for Complex Long-Horizon LLM Reasoning
  4. 2025Lost at the Beginning of Reasoning
  5. 2025Self-Consistency Is Losing Its Edge: Diminishing Returns and Rising Costs in Modern LLMs
  6. 2026Can We Break LLMs Out of Self-Loops? Fine-Grained Reasoning Control with Activation Steering
  7. 2025Entropy After </Think> for reasoning model early exiting
  8. 2026To Reason but Not to Stop: Understanding and Recovering Pruned Large Reasoning Models