# Is That Your Final Answer? Test-Time Scaling Improves Selective Question Answering

arXiv: https://arxiv.org/abs/2502.13962 · HTML: https://mlsanity.com/paper/2502.13962/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 18 of its 24 references that are in the map.
- First posted on arXiv 19 Feb 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 18 of 18)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [Selective Classification for Deep Neural Networks](https://mlsanity.com/paper/1705.08500/) (2017)
- [Measuring short-form factuality in large language models](https://mlsanity.com/paper/2411.04368/) (2024)
- [Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models](https://mlsanity.com/paper/2408.00724/) (2024)
- [Consistent Estimators for Learning to Defer to an Expert](https://mlsanity.com/paper/2006.01862/) (2020)
- [Selective Question Answering under Domain Shift](https://mlsanity.com/paper/2006.09462/) (2020)
- [Out-of-Distribution Detection and Selective Generation for Conditional Language Models](https://mlsanity.com/paper/2209.15558/) (2022)
- [Cost Sensitive Learning of Deep Feature Representations from Imbalanced Data](https://mlsanity.com/paper/1508.03422/) (2015)
- [Hypothesis Search: Inductive Reasoning with Language Models](https://mlsanity.com/paper/2309.05660/) (2023)
- [Learning to Defer to Multiple Experts: Consistent Surrogate Losses, Confidence Calibration, and Conformal Ensembles](https://mlsanity.com/paper/2210.16955/) (2022)
- [Combining Induction and Transduction for Abstract Reasoning](https://mlsanity.com/paper/2411.02272/) (2024)
- [Quizbowl: The Case for Incremental Question Answering](https://mlsanity.com/paper/1904.04792/) (2019)
- [Learn to Refuse: Making Large Language Models More Controllable and Reliable through Knowledge Scope Limitation and Refusal Mechanism](https://mlsanity.com/paper/2311.01041/) (2023)
- [What Question Answering can Learn from Trivia Nerds](https://mlsanity.com/paper/1910.14464/) (2019)

## Built on by (most cited first, 17 of 17)

- [Rank1: Test-Time Compute for Reranking in Information Retrieval](https://mlsanity.com/paper/2502.18418/) (2025)
- [Rank-K: Test-Time Reasoning for Listwise Reranking](https://mlsanity.com/paper/2505.14432/) (2025)
- [What Characterizes Effective Reasoning? Revisiting Length, Review, and Structure of CoT](https://mlsanity.com/paper/2509.19284/) (2025)
- [Entropy After </Think> for reasoning model early exiting](https://mlsanity.com/paper/2509.26522/) (2025)
- [When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling](https://mlsanity.com/paper/2604.10739/) (2026)
- [WikiVideo: Article Generation from Multiple Videos](https://mlsanity.com/paper/2504.00939/) (2025)
- [Read Your Own Mind: Reasoning Helps Surface Self-Confidence Signals in LLMs](https://mlsanity.com/paper/2505.23845/) (2025)
- [Conformal Thinking: Risk Control for Reasoning on a Compute Budget](https://mlsanity.com/paper/2602.03814/) (2026)
- [Always Tell Me The Odds: Fine-grained Conditional Probability Estimation](https://mlsanity.com/paper/2505.01595/) (2025)
- [FaithRL: Learning to Reason Faithfully through Step-Level Faithfulness Maximization](https://mlsanity.com/paper/2602.03507/) (2026)
- [Unified Multimodal Uncertain Inference](https://mlsanity.com/paper/2604.08701/) (2026)
- [Process Supervision of Confidence Margin for Calibrated LLM Reasoning](https://mlsanity.com/paper/2604.23333/) (2026)
- [Follow the Path: Reasoning over Knowledge Graph Paths to Improve Large Language Model Factuality](https://mlsanity.com/paper/2505.11140/) (2025)
- [Knowledge Fusion via Bidirectional Information Aggregation](https://mlsanity.com/paper/2507.08704/) (2025)
- [Seeing Through the MiRAGE: Evaluating Multimodal Retrieval Augmented Generation](https://mlsanity.com/paper/2510.24870/) (2025)
- [Configurable Reward Model for Balanced Safety Alignment](https://mlsanity.com/paper/2605.30487/) (2026)
- [FrankenReport: Early Exiting in Long-Form Generation Using Expected Value of Computation](https://mlsanity.com/paper/2609.06320/) (2026)

## Abstract (arXiv)

Scaling the test-time compute of large language models has demonstrated impressive performance on reasoning benchmarks. However, existing evaluations of test-time scaling make the strong assumption that a reasoning system should always give an answer to any question provided. This overlooks concerns about whether a model is confident in its answer, and whether it is appropriate to always provide a response. To address these concerns, we extract confidence scores during reasoning for thresholding model responses. We find that increasing compute budget at inference time not only helps models answer more questions correctly, but also increases confidence in correct responses. We then extend the current paradigm of zero-risk responses during evaluation by considering settings with non-zero levels of response risk, and suggest a recipe for reporting evaluations under these settings.