# Calibrate, Don't Curate: Label-Efficient Estimation from Noisy LLM Judges

arXiv: https://arxiv.org/abs/2605.09702 · HTML: https://mlsanity.com/paper/2605.09702/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 31 of its 46 references that are in the map.
- First posted on arXiv 10 May 2026.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Human values and AI feedback alignment](https://mlsanity.com/alignment/human-values-and-ai-feedback-alignment/)

## Builds on (most cited first, 30 of 31)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [Conformal Prediction Under Covariate Shift](https://mlsanity.com/paper/1904.06019/) (2019)
- [RewardBench: Evaluating Reward Models for Language Modeling](https://mlsanity.com/paper/2403.13787/) (2024)
- [Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge](https://mlsanity.com/paper/2410.02736/) (2024)
- [Conformal prediction beyond exchangeability](https://mlsanity.com/paper/2202.13415/) (2022)
- [JudgeBench: A Benchmark for Evaluating LLM-based Judges](https://mlsanity.com/paper/2410.12784/) (2024)
- [Verified Uncertainty Calibration](https://mlsanity.com/paper/1909.10155/) (2019)
- [Evaluating Large Language Models at Evaluating Instruction Following](https://mlsanity.com/paper/2310.07641/) (2023)
- [RewardBench 2: Advancing Reward Model Evaluation](https://mlsanity.com/paper/2506.01937/) (2025)
- [Trust or Escalate: LLM Judges with Provable Guarantees for Human Agreement](https://mlsanity.com/paper/2407.18370/) (2024)
- [Veridical Data Science](https://mlsanity.com/paper/1901.08152/) (2019)
- [Limits to scalable evaluation at the frontier: LLM as Judge won't beat twice the data](https://mlsanity.com/paper/2410.13341/) (2024)
- [Prediction-Powered Ranking of Large Language Models](https://mlsanity.com/paper/2402.17826/) (2024)
- [Adaptive conformal classification with noisy labels](https://mlsanity.com/paper/2309.05092/) (2023)
- [Label Noise Robustness of Conformal Prediction](https://mlsanity.com/paper/2209.14295/) (2022)
- [Overconfidence in LLM-as-a-Judge: Diagnosis and Confidence-Driven Solution](https://mlsanity.com/paper/2508.06225/) (2025)
- [Self-Calibrating Conformal Prediction](https://mlsanity.com/paper/2402.07307/) (2024)
- [Leveraging LLMs as Meta-Judges: A Multi-Agent Framework for Evaluating LLM Judgments](https://mlsanity.com/paper/2504.17087/) (2025)
- [QA-Calibration of Language Model Confidence Scores](https://mlsanity.com/paper/2410.06615/) (2024)
- [Calibrating LLM Judges: Linear Probes for Fast and Reliable Uncertainty Estimation](https://mlsanity.com/paper/2512.22245/) (2025)
- [Generalized Venn and Venn-Abers Calibration with Applications in Conformal Prediction](https://mlsanity.com/paper/2502.05676/) (2025)
- [A Judge-Aware Ranking Framework for Evaluating Large Language Models without Ground Truth](https://mlsanity.com/paper/2601.21817/) (2026)
- [Who can we trust? LLM-as-a-jury for Comparative Assessment](https://mlsanity.com/paper/2602.16610/) (2026)
- [Quantitative LLM Judges](https://mlsanity.com/paper/2506.02945/) (2025)
- [CLEAR: Calibrated Learning for Epistemic and Aleatoric Risk](https://mlsanity.com/paper/2507.08150/) (2025)
- [PRECISE: Reducing the Bias of LLM Evaluations Using Prediction-Powered Ranking Estimation](https://mlsanity.com/paper/2601.18777/) (2026)
- [Efficient Bayesian Inference from Noisy Pairwise Comparisons](https://mlsanity.com/paper/2510.09333/) (2025)

## Built on by (most cited first, 5 of 5)

- [A Finite-Calibration Regime Map for LLM Judge Panels](https://mlsanity.com/paper/2606.01034/) (2026)
- [Stopping and Routing LLM Judge Panels](https://mlsanity.com/paper/2608.19802/) (2026)
- [Aggregate Disambiguation Systems](https://mlsanity.com/paper/2608.30805/) (2026)
- [RAIM: Robust Aggregation of Inexpensive Models for Hallucination Detection](https://mlsanity.com/paper/2609.39229/) (2026)
- [Are We Testing Too Much? A Survey of Benchmark Compression for Efficient Evaluation of Large Language Models](https://mlsanity.com/paper/2610.18248/) (2026)

## Abstract (arXiv)

Multi-judge evaluation is increasingly used to assess LLMs and reward models, and the prevailing heuristic is to curate: keep the most accurate judges and discard weaker ones. We show that this heuristic can reverse when the target is not point accuracy, but calibrated probabilistic evaluation from a labeled calibration set. Holding the aggregation and calibration procedures fixed, we compare accuracy-ranked top-k judge selection with using the full judge panel. Across four labeled pairwise-evaluation benchmarks spanning LLM-as-judge and reward-model settings, the calibrated full panel consistently outperforms accuracy-based selection. On RewardBench2, retaining all judges achieves negative log-likelihood (NLL) of 0.006 versus 0.013 under top-5 selection, halving the calibration error. This advantage persists after judge-family deduplication and against stronger same-pipeline subset search. We explain this reversal with oracle analyses showing that the optimal calibrated risk under proper scoring rules cannot increase when additional judge signals are made available, and that even below-chance judges can be useful when their biases are learnable and their signals are non-redundant. The resulting operating principle is simple: in multi-judge evaluation with labeled calibration data, do not discard weak judges by accuracy alone; keep them when they are parseable, non-redundant, and calibratable.