mlsanityData to 14 Oct 2026

Learning from the Gap Between Pass@K and Pass@1

arXiv 2609.35793 ↗First posted 17 Sep 2026cs.LG stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Sampling many responses and keeping one that passes a verifier lets large language models solve problems beyond their single-response ability, but this search must be paid again for every query, while many deployments answer with a single response. Post-training on verified responses can transfer the benefit of search into the model. With a fixed budget, selecting by correctness alone spends slots on problems the model already answers correctly, leaving fewer to correct its failures. To address this imbalance, we propose GapFT, which trains on the gap between Pass@K and Pass@1: problems that the source model fails with one response but solves within K samples. GapFT keeps the objective and training budget fixed and changes only which verified responses enter training; an exact decomposition splits the resulting Pass@1 change into corrected failures and regressions on problems the source model already solved. On LogiQA 2.0 and ReClor with three model families, GapFT is above budget-matched uniform rejection-sampling fine-tuning (RFT) in every setting, with a positive pooled effect, and on Llama-3.1-8B and Mistral-7B it recovers about two thirds to four fifths of the gain of fine-tuning on the entire verified pool with 11-34% of its problems. Further analyses reveal that the gain comes from failures that the first few search samples recover, while failures found only by deeper search displace replay and add no net gain, that filling the same budget with gold-labeled failures search cannot reach lowers accuracy, and that the gain is bounded by how many transferable failures search exposes.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 36 of its 41 references that are in the map. First posted on arXiv 17 Sep 2026.

Builds on

36 of 41 references.

References · 36

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  9. 2024Qwen2.5 Technical Reportcited by 4,942
  10. 2023Let's Verify Step by Stepcited by 3,789
  11. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  12. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
24 more
  1. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  2. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  3. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  4. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  5. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
  6. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  7. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 500
  8. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 427
  9. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 426
  10. 2024ReFT: Reasoning with Reinforced Fine-Tuningcited by 239
  11. 2020ReClor: A Reading Comprehension Dataset Requiring Logical Reasoningcited by 217
  12. 2024V-STaR: Training Verifiers for Self-Taught Reasonerscited by 217
  13. 2023Mitigating the Alignment Tax of RLHFcited by 183
  14. 2025Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretrainingcited by 129
  15. 2024DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solvingcited by 95
  16. 2024Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Modelscited by 89
  17. 2025Scaling Test-Time Compute Without Verification or RL is Suboptimalcited by 84
  18. 2024BOND: Aligning LLMs with Best-of-N Distillationcited by 76
  19. 2024Variational Best-of-N Alignmentcited by 51
  20. 2025Prompt Curriculum Learning for Efficient LLM Post-Trainingcited by 45
  21. 2025From f(x) and g(x) to f(g(x)): LLMs Learn New Skills in RL by Composing Old Onescited by 42
  22. 2024Scaling Laws for Forgetting When Fine-Tuning Large Language Modelscited by 38
  23. 2024ReGenesis: LLMs can Grow into Reasoning Generalists via Self-Improvementcited by 18
  24. 2026Why Pass@k Optimization Can Degrade Pass@1: Prompt Interference in LLM Post-trainingcited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Consolidating Rewarded Perturbations for LLM Post-Training
  2. 2026Free Lunch for Pass@k? Low Cost Diverse Sampling for Diffusion Language Models
  3. 2025Gaperon: A Peppered English-French Generative Language Model Suite
  4. 2025Rethinking Cross-lingual Gaps from a Statistical Viewpoint
  5. 2025Uncovering Competency Gaps in Large Language Models and Their Benchmarks
  6. 2026Probe the Harness: Setup Checks for Stale-Data RL Comparisons in Language Models
  7. 2025Finish First, Perfect Later: Test-Time Token-Level Cross-Validation for Diffusion Large Language Models
  8. 2025Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs