mlsanityData to 14 Oct 2026

Scaling Evaluation-time Compute with Reasoning Models as Evaluators

arXiv 2503.19877 ↗First posted 25 Mar 2025cs.CLcited by 13

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

As language model (LM) outputs get more and more natural, it is becoming more difficult than ever to evaluate their quality. Simultaneously, increasing LMs' "thinking" time through scaling test-time compute has proven an effective technique to solve challenging problems in domains such as math and code. This raises a natural question: can an LM's evaluation capability also be improved by spending more test-time compute? To answer this, we investigate employing reasoning models-LMs that natively generate long chain-of-thought reasoning-as evaluators. Specifically, we examine methods to leverage more test-time compute by (1) using reasoning models, and (2) prompting these models to evaluate not only the response as a whole (i.e., outcome evaluation) but also assess each step in the response separately (i.e., process evaluation). In experiments, we observe that the evaluator's performance improves monotonically when generating more reasoning tokens, similar to the trends observed in LM-based generation. Furthermore, we use these more accurate evaluators to rerank multiple generations, and demonstrate that spending more compute at evaluation time can be as effective as using more compute at generation time in improving an LM's problem-solving capability.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 34 of its 76 references that are in the map. First posted on arXiv 25 Mar 2025.

Builds on

34 of 76 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning6 Mar
  2. 2025Linguistic Generalizability of Test-Time Scaling in Mathematical Reasoning24 Feb
  3. 2025s1: Simple test-time scaling31 Jan
  4. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug

Other references · 30

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2020Scaling Laws for Neural Language Modelscited by 6,586
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  6. 2024A Survey on LLM-as-a-Judgecited by 1,423
  7. 2024OpenAI o1 System Cardcited by 1,244
  8. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  9. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  10. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  11. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  12. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 897
18 more
  1. 2025Process Reinforcement through Implicit Rewardscited by 504
  2. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 499
  3. 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 446
  4. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoningcited by 415
  5. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 377
  6. 2024Skywork-Reward: Bag of Tricks for Reward Modeling in LLMscited by 327
  7. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 293
  8. 2024ProcessBench: Identifying Process Errors in Mathematical Reasoningcited by 292
  9. 2024RLHF Workflow: From Reward Modeling to Online RLHFcited by 267
  10. 2025PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Modelscited by 99
  11. 2024The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Modelscited by 93
  12. 2024A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chainscited by 65
  13. 2025Evaluating Step-by-step Reasoning Traces: A Surveycited by 63
  14. 2023MR-GSM8K: A Meta-Reasoning Benchmark for Large Language Model Evaluationcited by 57
  15. 2023The Alignment Ceiling: Objective Mismatch in Reinforcement Learning from Human Feedbackcited by 42
  16. 2024Better Instruction-Following Through Minimum Bayes Riskcited by 19
  17. 2025Examining False Positives under Inference Scaling for Mathematical Reasoningcited by 11
  18. 2025Verdict: A Library for Scaling Judge-Time Computecited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 7.

13 papers.

Sort:

By area:

  1. 2025Inference-Time Scaling for Generalist Reward Modelingcited by 239
  2. 2025A Survey of Process Reward Models: From Outcome Signals to Process Supervisions for Large Language Modelscited by 38
  3. 2026JEV-as-a-Judge: Accept When Confident, Escalate When Unsurecited by 30
  4. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 15
  5. 2026InT: Self-Proposed Interventions Enable Credit Assignment in LLM Reasoningcited by 14
  6. 2026Examining Reasoning LLMs-as-Judges in Non-Verifiable LLM Post-Trainingcited by 13
  7. 2025ReasoningFlow: Semantic Structure of Complex Reasoning Tracescited by 12
  8. 2025Enhancing Large Language Model Reasoning with Reward Models: An Analytical Surveycited by 8
  9. 2025Evaluating Legal Reasoning Traces with Legal Issue Tree Rubricscited by 5
  10. 2026ReasoningFlow: Discourse Structures for Understanding LLM Reasoning Tracescited by 5
  11. 2026Reasoning over mathematical objects: on-policy reward modeling and test time aggregationcited by 3
  12. 2026Agent Step Value: Auditing Evaluation-Channel Sensitivity in Black-Box Agent Tracescited by 0
1 more
  1. 2026CoBa: Cost-Effective Test-Time Scaling via Compute-Balanced Routingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators
  2. 2026When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling
  3. 2025Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning
  4. 2025The Art of Scaling Test-Time Compute for Large Language Models
  5. 2026ThinkBooster: A Unified Framework for Seamless Test-Time Scaling of LLM Reasoning
  6. 2025An Empirical Study of Reasoning Steps in Thinking Code LLMs
  7. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?
  8. 2025Inference-Time Scaling for Complex Tasks: Where We Stand and What Lies Ahead