mlsanityData to 13 Oct 2026

Learning Generative Selection for Best-of-N

arXiv 2602.02143 ↗First posted 2 Feb 2026cs.LG cs.AI cs.CLcited by 2

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Scaling test-time compute via parallel sampling can substantially improve LLM reasoning, but is often limited by Best-of-N selection quality. Generative selection methods, such as GenSelect, address this bottleneck, yet strong selection performance remains largely limited to large models. We show that small reasoning models can acquire strong GenSelect capabilities through targeted reinforcement learning. To this end, we synthesize selection tasks from large-scale math and code instruction datasets by filtering to instances with both correct and incorrect candidate solutions, and train 1.7B-parameter models with DAPO to reward correct selections. Across math (AIME24, AIME25, HMMT25) and code (LiveCodeBench) reasoning benchmarks, our models consistently outperform prompting and majority-voting baselines, often approaching or exceeding much larger models. Moreover, these gains generalize to selecting outputs from stronger models despite training only on outputs from weaker models. Overall, our results establish reinforcement learning as a scalable way to unlock strong generative selection in small models, enabling efficient test-time scaling.

Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 21 of its 27 references that are in the map. First posted on arXiv 2 Feb 2026.

Builds on

21 of 27 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025The Majority is not always right: RL training for solution aggregation8 Sep
  2. 2025AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learning22 May
  3. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  4. 2025Scoring Verifiers: Evaluating Synthetic Verification for Code and Reasoning19 Feb
  5. 2024Generative Verifiers: Reward Modeling as Next-Token Prediction27 Aug

Other references · 16

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,916
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,657
  3. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,003
  4. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,954
  5. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 970
  6. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 220
  7. 2024OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Datacited by 197
  8. 2025ACECODER: Acing Coder RL via Automated Test-Case Synthesiscited by 91
  9. 2024AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward Modelingcited by 77
  10. 2025Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitatecited by 61
  11. 2025Sample, Scrutinize and Scale: Effective Inference-Time Search by Scaling Verificationcited by 49
  12. 2025CodeContests+: High-Quality Test Case Generation for Competitive Programmingcited by 41
4 more
  1. 2025Learning to Reason Across Parallel Samples for LLM Reasoningcited by 32
  2. 2025Heimdall: test-time scaling on the generative verificationcited by 26
  3. 2025OpenCodeReasoning-II: A Simple Test Time Scaling Approach via Self-Critiquecited by 24
  4. 2025Learning to Refine: Self-Refinement of Parallel Reasoning in LLMscited by 12

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Primal Generation, Dual Judgment: Self-Training from Test-Time Scalingcited by 0
  2. 2026Selection, Recombination, or a Fresh Solve? A Candidate-Free Control for Single-Pass Test-Time Aggregationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025GenSelect: A Generative Approach to Best-of-N
  2. 2026Back to Basics: Revisiting Exploration in Reinforcement Learning for LLM Reasoning via Generative Probabilities
  3. 2025Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
  4. 2025T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling
  5. 2025Incentivizing LLMs to Self-Verify Their Answers
  6. 2025When To Solve, When To Verify: Compute-Optimal Problem Solving and Generative Verification for LLM Reasoning
  7. 2025TTRL: Test-Time Reinforcement Learning
  8. 2025InternBootcamp: Boosting LLM Reasoning with Verifiable Task Scaling