mlsanityData to 7 Oct 2026

Re-FORC: Adaptive Reward Prediction for Efficient Chain-of-Thought Reasoning

arXiv 2511.02130 ↗First posted 3 Nov 2025cs.AI cs.LGcited by 4

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

We propose Re-FORC, an adaptive reward prediction method that, given a query, enables prediction of the expected future rewards as a function of the number of future thinking tokens. Re-FORC trains a lightweight adapter on reasoning models, demonstrating improved prediction with longer reasoning and larger models. Re-FORC enables: 1) early stopping of unpromising reasoning chains, reducing compute by up to 26% compared to fixed-budget cutoffs, while maintaining accuracy, 2) optimized model and thinking length selection that outperforms the largest model alone--- reaching 1.7 percentage points higher peak accuracy while needing up to 12% less compute to match the largest model's accuracy, 3) adaptive test-time scaling, which increases accuracy by 9.9 percentage points (on average at maximum compute) over confidence-based baselines. Re-FORC allows dynamic reasoning with length control via cost-per-token thresholds while estimating computation time upfront.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 42 of its 58 references that are in the map. First posted on arXiv 3 Nov 2025.

Builds on

42 of 58 references.

Closest ancestors · 20

Same direction, newest first

  1. 2025e1: Learning Adaptive Control of Reasoning Effort30 Oct
  2. 2025Stop When Enough: Adaptive Early-Stopping for Chain-of-Thought Reasoning11 Oct
  3. 2025Adaptive Test-Time Reasoning via Reward-Guided Dual-Phase Search29 Sep
  4. 2025LATTS: Locally Adaptive Test-Time Scaling16 Sep
  5. 2025Deep Think with Confidence21 Aug
  6. 2025Can Past Experience Accelerate LLM Reasoning?27 May
  7. 2025Reasoning Models Better Express Their Confidence20 May
  8. 2025Dynamic Early Exit in Reasoning Models22 Apr
  9. 2025Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification7 Apr
  10. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning6 Mar
  11. 2025Don't Get Lost in the Trees: Streamlining LLM Reasoning by Overcoming Tree Search Exploration Pitfalls16 Feb
  12. 2025Confidence Improves Self-Consistency in LLMs10 Feb
8 more
  1. 2025s1: Simple test-time scaling31 Jan
  2. 2024Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning12 Dec
  3. 2024Learning How Hard to Think: Input-Adaptive Allocation of LM Computation7 Oct
  4. 2024Adaptive Inference-Time Compute: LLMs Can Predict if They Can Do Better, Even Mid-Generation3 Oct
  5. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
  6. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models1 Aug
  7. 2024Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning19 Jan
  8. 2023Large Language Model Guided Tree-of-Thought15 May

Other references · 22

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  5. 2023Let's Verify Step by Stepcited by 3,598
  6. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  7. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  8. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 550
  9. 2022Confident Adaptive Language Modelingcited by 297
  10. 2025Universal Model Routing for Efficient LLM Inferencecited by 80
  11. 2024Demystifying Chains, Trees, and Graphs of Thoughtscited by 63
  12. 2025BEST-Route: Adaptive LLM Routing with Test-Time Optimal Computecited by 63
10 more
  1. 2025Answer Convergence as a Signal for Early Stopping in Reasoningcited by 38
  2. 2024Smoothie: Label Free Language Model Routingcited by 31
  3. 2025Learning When to Plan: Efficiently Allocating Test-Time Compute for LLM Agentscited by 18
  4. 2024Cost-aware Bayesian Optimization via the Pandora's Box Gittins Indexcited by 16
  5. 2025Are the Hidden States Hiding Something? Testing the Limits of Factuality-Encoding Capabilities in LLMscited by 11
  6. 2025AI Agents as Universal Task Solverscited by 9
  7. 2025When Do LLMs Admit Their Mistakes? Understanding The Role Of Model Belief In Retractioncited by 6
  8. 2025The Gittins Index: A Design Principle for Decision-Making Under Uncertaintycited by 6
  9. 2025The Pandora's Box Problem with Sequential Inspectionscited by 3
  10. 2025Best-of-∞ -- Asymptotic Performance of Test-Time LLM Ensemblingcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Reasoning about Reasoning: BAPO Bounds on Chain-of-Thought Token Complexity in LLMscited by 1
  2. 2026Granularity-Regulated Adaptive Computational Efficiency for Optimal Verification in Test-Time Scalingcited by 1
  3. 2026Cache-Aware Prompt Compression:A Two-Tier Cost Model for LLM API Cachingcited by 1
  4. 2026ExecTune: Effective Steering of Black-Box LLMs with Guide Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Reasoning without Regret
  2. 2025Rethinking Reasoning Quality in Large Language Models through Enhanced Chain-of-Thought via RL
  3. 2026On the Power of (Approximate) Reward Models for Inference-Time Scaling
  4. 2025Reward Reasoning Model
  5. 2026Reinforcing Chain-of-Thought Reasoning with Self-Evolving Rubrics
  6. 2026Nice Fold or Hero Call: Learning Budget-Efficient Thinking for Adaptive Reasoning
  7. 2026Latent Reward Steering: An Adaptive Inference-Time Framework that Implicitly Promotes Cognitive Behaviors in Reasoning LLMs
  8. 2025When to Continue Thinking: Adaptive Thinking Mode Switching for Efficient Reasoning