# Why Pass@k Optimization Can Degrade Pass@1: Prompt Interference in LLM Post-training

arXiv: https://arxiv.org/abs/2602.21189 · HTML: https://mlsanity.com/paper/2602.21189/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 31 of its 34 references that are in the map.
- First posted on arXiv 24 Feb 2026.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)

## Builds on (most cited first, 30 of 31)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [Solving math word problems with process- and outcome-based feedback](https://mlsanity.com/paper/2211.14275/) (2022)
- [SPoC: Search-based Pseudocode to Code](https://mlsanity.com/paper/1906.04908/) (2019)
- [RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning](https://mlsanity.com/paper/2410.02089/) (2024)
- [A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforce](https://mlsanity.com/paper/2504.11343/) (2025)
- [Pass@k Training for Adaptively Balancing Exploration and Exploitation of Large Reasoning Models](https://mlsanity.com/paper/2508.10751/) (2025)
- [Theoretical guarantees on the best-of-n alignment policy](https://mlsanity.com/paper/2401.01879/) (2024)
- [Reinforcement Learning with Verifiable Rewards: GRPO's Effective Loss, Dynamics, and Success Amplification](https://mlsanity.com/paper/2503.06639/) (2025)
- [Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Models](https://mlsanity.com/paper/2412.15287/) (2024)
- [Pass@K Policy Optimization: Solving Harder Reinforcement Learning Problems](https://mlsanity.com/paper/2505.15201/) (2025)
- [A general sample complexity analysis of vanilla policy gradient](https://mlsanity.com/paper/2107.11433/) (2021)
- [BOND: Aligning LLMs with Best-of-N Distillation](https://mlsanity.com/paper/2407.14622/) (2024)
- [Optimizing Language Models for Inference Time Objectives using Reinforcement Learning](https://mlsanity.com/paper/2503.19595/) (2025)
- [Variational Best-of-N Alignment](https://mlsanity.com/paper/2407.06057/) (2024)
- [Ray Interference: a Source of Plateaus in Deep Reinforcement Learning](https://mlsanity.com/paper/1904.11455/) (2019)
- [InfAlign: Inference-aware language model alignment](https://mlsanity.com/paper/2412.19792/) (2024)
- [Optimizing Temperature for Language Models with Multi-Sample Inference](https://mlsanity.com/paper/2502.05234/) (2025)
- [On the Effect of Negative Gradient in Group Relative Deep Reinforcement Optimization](https://mlsanity.com/paper/2505.18830/) (2025)
- [Smoothing Policies and Safe Policy Gradients](https://mlsanity.com/paper/1905.03231/) (2019)
- [What is the objective of reasoning with reinforcement learning?](https://mlsanity.com/paper/2510.13651/) (2025)
- [Beyond the Sampled Token: Preserving Candidate Support in RLVR](https://mlsanity.com/paper/2510.14807/) (2025)
- [Best-of-N through the Smoothing Lens: KL Divergence and Regret Analysis](https://mlsanity.com/paper/2507.05913/) (2025)
- [Risk-Sensitive RL for Alleviating Exploration Dilemmas in Large Language Models](https://mlsanity.com/paper/2509.24261/) (2025)
- [Pass@k Metric for RLVR: A Diagnostic Tool of Exploration, But Not an Objective](https://mlsanity.com/paper/2511.16231/) (2025)
- [Advantage Shaping as Surrogate Reward Maximization: Unifying Pass@K Policy Gradients](https://mlsanity.com/paper/2510.23049/) (2025)

## Built on by (most cited first, 8 of 8)

- [Beyond the Mean: Multi-Moment Policy Optimization for LLM Reasoning](https://mlsanity.com/paper/2608.02149/) (2026)
- [CurveRL: Principled Distribution-Aware Context Reweighting for LLM Reasoning](https://mlsanity.com/paper/2605.24331/) (2026)
- [Don't Let Gains FADE: Breaking Down Policy Gradient Weights in RL](https://mlsanity.com/paper/2607.01490/) (2026)
- [When Does the Best Sampling Temperature Rise with the Budget? Sufficient Conditions for Pass@k](https://mlsanity.com/paper/2608.14665/) (2026)
- [GUPO: Gradient Uncertainty-aware Policy Optimization for Post-Training Large Language Models](https://mlsanity.com/paper/2608.17411/) (2026)
- [Group Adaptive Clipping Policy Optimization](https://mlsanity.com/paper/2609.00444/) (2026)
- [Learning from the Gap Between Pass@K and Pass@1](https://mlsanity.com/paper/2609.35793/) (2026)
- [SERA: Scale-Equalized Rollout Allocation for Maximum Likelihood Reinforcement Learning](https://mlsanity.com/paper/2609.36552/) (2026)

## Abstract (arXiv)

Pass@k is a widely used performance metric for verifiable large language model tasks, including mathematical reasoning, code generation, and short-answer reasoning. It defines success if any of k independently sampled solutions passes a verifier. This multi-sample inference metric has motivated inference-aware fine-tuning methods that directly optimize pass@k. However, prior work reports a trade-off: pass@k improves while pass@1 degrades under such methods. This trade-off is practically important because pass@1 often remains a hard operational constraint due to latency and cost budgets, imperfect verifier coverage, and the need for a reliable single-shot fallback. We study the origin of this trade-off and provide a theoretical characterization of when pass@k policy optimization can reduce pass@1 through gradient conflict induced by prompt interference. This interference arises because shared model parameters couple prompts, so improving performance on some prompts can adversely affect others. We show that pass@k policy gradients can conflict with pass@1 gradients because pass@k optimization implicitly reweights prompts toward low-success prompts; when these prompts are negatively interfering, their upweighting can rotate the pass@k update direction away from the pass@1 direction. We illustrate our findings with large language model experiments on verifiable mathematical reasoning and coding tasks.