# More Bang for the Buck: Improving the Inference of Large Language Models at a Fixed Budget using Reset and Discard (ReD)

arXiv: https://arxiv.org/abs/2601.21522 · HTML: https://mlsanity.com/paper/2601.21522/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 69 references that are in the map.
- First posted on arXiv 29 Jan 2026.

## Builds on (most cited first, 28 of 28)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [Large Language Monkeys: Scaling Inference Compute with Repeated Sampling](https://mlsanity.com/paper/2407.21787/) (2024)
- [STaR: Bootstrapping Reasoning With Reasoning](https://mlsanity.com/paper/2203.14465/) (2022)
- [FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance](https://mlsanity.com/paper/2305.05176/) (2023)
- [DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving](https://mlsanity.com/paper/2401.09670/) (2024)
- [Reinforced Self-Training (ReST) for Language Modeling](https://mlsanity.com/paper/2308.08998/) (2023)
- [Scaling Data-Constrained Language Models](https://mlsanity.com/paper/2305.16264/) (2023)
- [SPoC: Search-based Pseudocode to Code](https://mlsanity.com/paper/1906.04908/) (2019)
- [From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models](https://mlsanity.com/paper/2406.16838/) (2024)
- [Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Sampling](https://mlsanity.com/paper/2408.16737/) (2024)
- [Code Repair with LLMs gives an Exploration-Exploitation Tradeoff](https://mlsanity.com/paper/2405.17503/) (2024)
- [CodeMonkeys: Scaling Test-Time Compute for Software Engineering](https://mlsanity.com/paper/2501.14723/) (2025)
- [Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategies](https://mlsanity.com/paper/2406.06461/) (2024)
- [The Larger the Better? Improved LLM Code-Generation via Budget Reallocation](https://mlsanity.com/paper/2404.00725/) (2024)
- [How Do Large Language Monkeys Get Their Power (Laws)?](https://mlsanity.com/paper/2502.17578/) (2025)
- [A Simple Model of Inference Scaling Laws](https://mlsanity.com/paper/2410.16377/) (2024)
- [To Backtrack or Not to Backtrack: When Sequential Search Limits Model Reasoning](https://mlsanity.com/paper/2504.07052/) (2025)
- [Efficient Prediction of Pass@k Scaling in Large Language Models](https://mlsanity.com/paper/2510.05197/) (2025)
- [Mitigating Tail Narrowing in LLM Self-Improvement via Socratic-Guided Sampling](https://mlsanity.com/paper/2411.00750/) (2024)
- [DISC: Dynamic Decomposition Improves LLM Inference Scaling](https://mlsanity.com/paper/2502.16706/) (2025)
- [Stochastic Resetting Mitigates Latent Gradient Bias of SGD from Label Noise](https://mlsanity.com/paper/2406.00396/) (2024)
- [First-Passage Approach to Optimizing Perturbations for Improved Training of Machine Learning Models](https://mlsanity.com/paper/2502.04121/) (2025)

## Built on by (most cited first, 2 of 2)

- [Compute Allocation for Self-Evolving LLMs: From Depth-Breadth to Multi-Armed Bandits](https://mlsanity.com/paper/2605.29268/) (2026)
- [When Does Longer Reasoning Help? Predicting Mathematical Reasoning Through Discovery and Execution](https://mlsanity.com/paper/2610.05322/) (2026)

## Abstract (arXiv)

The performance of large language models (LLMs) on verifiable tasks is usually measured by pass@k, the probability of answering a question correctly at least once in k trials. At a fixed budget, a more suitable metric is coverage@cost, the average number of unique questions answered as a function of the total number of attempts. We connect the two metrics and show that the empirically-observed power-law behavior in pass@k leads to a sublinear growth of the coverage@cost (diminishing returns). To solve this problem, we propose Reset-and-Discard (ReD), a query method of LLMs that increases coverage@cost for a given budget, regardless of the pass@k form. Moreover, given a pass@k, we can quantitatively predict the savings in the total number of attempts using ReD. If pass@k is not available for the model, ReD can infer its power-law exponent. Experiments on three LLMs across coding (HumanEval), math (GSM8K), and reasoning (MMLU-Pro) benchmarks demonstrate that ReD substantially reduces the required attempts, tokens, and USD cost to reach a desired coverage, while also offering an efficient way to measure inference power-laws. ReD's advantage is maintained for imperfect verifiers and outperforms the tested allocation baselines.