mlsanityData to 7 Oct 2026

SCATR: Simple Calibrated Test-Time Ranking

arXiv 2604.16535 ↗First posted 16 Apr 2026cs.LG cs.AIcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Test-time scaling (TTS) improves large language models (LLMs) by allocating additional compute at inference time. In practice, TTS is often achieved through parallel scaling: generating multiple candidate responses and selecting the best via a Best-of-N (BoN) strategy. Its effectiveness therefore hinges on the scoring function. Learned scorers such as process reward models (PRMs) can be strong, but they are expensive to train and run. Lightweight confidence heuristics based on token log-probabilities are much cheaper, yet we find that they often perform substantially worse. To improve on lightweight confidence heuristics without incurring the full cost of stronger learned scorers, we introduce SCATR, a simple and efficient BoN ranking method that learns a lightweight scorer from a small calibration set using hidden representations from the base model. Across coding and mathematical reasoning benchmarks, SCATR improves over prior confidence-based baselines by up to 9%. Relative to LoRA fine-tuning on the same calibration data, it achieves comparable accuracy with up to 8000x fewer trainable parameters and much lower compute, reducing training and inference latency by up to 150x and 1000x, respectively. SCATR is also competitive with strong PRM baselines, and in several settings improves accuracy by up to 7.8% on math and 4.2% on coding while enabling up to 1000x faster inference. Overall, SCATR offers a strong accuracy-efficiency trade-off for scalable test-time selection.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 27 of its 36 references that are in the map. First posted on arXiv 16 Apr 2026.

Builds on

27 of 36 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Learning to Refine: Self-Refinement of Parallel Reasoning in LLMs27 Aug
  2. 2025Efficient Test-Time Scaling via Self-Calibration25 Feb
  3. 2025Scalable Best-of-N Selection for Large Language Models via Self-Certainty25 Feb
  4. 2025s1: Simple test-time scaling31 Jan
  5. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
  6. 2023Universal Self-Consistency for Large Language Model Generation29 Nov
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May
  8. 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar

Other references · 19

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  4. 2023Teaching Large Language Models to Self-Debugcited by 963
  5. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  6. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
  7. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  8. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 317
  9. 2024Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantificationcited by 159
  10. 2025Is Best-of-N the Best of Them? Coverage, Scaling, and Optimality in Inference-Time Alignmentcited by 91
  11. 2024Process Reward Model with Q-Value Rankingscited by 78
  12. 2023Self-Evaluation Improves Selective Generation in Large Language Modelscited by 74
7 more
  1. 2024Learning to Route LLMs with Confidence Tokenscited by 66
  2. 2025The Majority is not always right: RL training for solution aggregationcited by 46
  3. 2025ReasonFlux-PRM: Trajectory-Aware PRMs for Long Chain-of-Thought Reasoning in LLMscited by 44
  4. 2025LLMs Can Generate a Better Answer by Aggregating Their Own Responsescited by 27
  5. 2025Scalable In-context Ranking with Generative Modelscited by 8
  6. 2025LLMs are Overconfident: Evaluating Confidence Interval Calibration with FermiEvalcited by 8
  7. 2025Optimal Aggregation of LLM and PRM Signals for Efficient Test-Time Scalingcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Trust but Verify! A Survey on Verification Design for Test-time Scaling
  2. 2025The Art of Scaling Test-Time Compute for Large Language Models
  3. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models
  4. 2026Scaling with Confidence: Calibrating Confidence of LLMs for Adaptive Test Time Scaling
  5. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?
  6. 2025First Finish Search: Efficient Test-Time Scaling in Large Language Models
  7. 2026Predicting and improving test-time scaling laws via reward tail-guided search
  8. 2026Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility