# PairJudge RM: Perform Best-of-N Sampling with Knockout Tournament

arXiv: https://arxiv.org/abs/2501.13007 · HTML: https://mlsanity.com/paper/2501.13007/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- First posted on arXiv 22 Jan 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)
- LLM alignment / RLHF / preference optimisation / [Process reward models for reasoning](https://mlsanity.com/alignment/process-reward-models-for-reasoning/)

## Built on by (most cited first, 22 of 22)

- [Inference-Time Scaling for Generalist Reward Modeling](https://mlsanity.com/paper/2504.02495/) (2025)
- [OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignment](https://mlsanity.com/paper/2510.07743/) (2025)
- [A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems](https://mlsanity.com/paper/2504.09037/) (2025)
- [Reward Reasoning Model](https://mlsanity.com/paper/2505.14674/) (2025)
- [Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators](https://mlsanity.com/paper/2504.15253/) (2025)
- [Think When You Need: Self-Adaptive Chain-of-Thought Learning](https://mlsanity.com/paper/2504.03234/) (2025)
- [The Open Proof Corpus: A Large-Scale Study of LLM-Generated Mathematical Proofs](https://mlsanity.com/paper/2506.21621/) (2025)
- [Robust Reward Modeling via Causal Rubrics](https://mlsanity.com/paper/2506.16507/) (2025)
- [From What to Why: A Multi-Agent System for Evidence-based Chemical Reaction Condition Reasoning](https://mlsanity.com/paper/2509.23768/) (2025)
- [A Survey on Parallel Reasoning](https://mlsanity.com/paper/2510.12164/) (2025)
- [Scaling Test-Time Compute to Achieve IOI Gold Medal with Open-Weight Models](https://mlsanity.com/paper/2510.14232/) (2025)
- [Uncertainty Quantification for Large Language Model Reward Learning under Heterogeneous Human Feedback](https://mlsanity.com/paper/2512.03208/) (2025)
- [Enhancing Large Language Model Reasoning with Reward Models: An Analytical Survey](https://mlsanity.com/paper/2510.01925/) (2025)
- [WebArbiter: A Principle-Guided Reasoning Process Reward Model for Web Agents](https://mlsanity.com/paper/2601.21872/) (2026)
- [Reinforcement Learning from Human Feedback: A Statistical Perspective](https://mlsanity.com/paper/2604.02507/) (2026)
- [Z-Reward: Beyond Scalar Rewards by Internalizing Reasoning into Score Distributions](https://mlsanity.com/paper/2606.09076/) (2026)
- [TextBO: Bayesian Optimization in Language Space for Eval-Efficient Self-Improving AI](https://mlsanity.com/paper/2511.12063/) (2025)
- [A Survey on Self-Improving Test-Time Intelligence: Feedback-Driven Adapting, Learning, and Scaling at Inference](https://mlsanity.com/paper/2609.01679/) (2026)
- [T-pro 2.0: An Efficient Russian Hybrid-Reasoning Model and Playground](https://mlsanity.com/paper/2512.10430/) (2025)
- [GRRM: Group Relative Reward Modeling for Machine Translation](https://mlsanity.com/paper/2602.14028/) (2026)
- [RETuning: Upgrading Inference-Time Scaling for Stock Movement Prediction with Large Language Models](https://mlsanity.com/paper/2510.21604/) (2025)
- [DAJ: Data-Reweighted LLM Judge for Test-Time Scaling in Code Generation](https://mlsanity.com/paper/2601.22230/) (2026)

## Abstract (arXiv)

Best-of-N (BoN) sampling, a common strategy for test-time scaling of Large Language Models (LLMs), relies on reward models to select the best candidate solution from multiple generations. However, traditional reward models often assign arbitrary and inconsistent scores, limiting their effectiveness. To address this, we propose a Pairwise Judge Reward Model (PariJudge RM) combined with a knockout tournament for BoN sampling. Instead of assigning absolute scores, given one math problem, PariJudge RM judges two candidate solutions' correctness with chain-of-thought reasoning simultaneously. This approach eliminates the need for scoring and enables cross-validation of solutions through parallel judgment. In the knockout tournament, PariJudge RM conducts pairwise Judgment between candidate solutions and eliminates the incorrect ones iteratively. We construct PairJudge-432K, a large-scale dataset of 432K pairwise judgments derived from NumiaMath and annotated using textttgemini-1.5-flash, and train the PariJudge RM via supervised fine-tuning. Experiments on MATH-500 and the Olympiad Bench demonstrate significant improvements over baseline reward models. And a 40% to 60% relative improvement is achieved on the top 50% challenging problems.