mlsanityData to 7 Oct 2026

Calibrate, Don't Curate: Label-Efficient Estimation from Noisy LLM Judges

arXiv 2605.09702 ↗First posted 10 May 2026stat.ME cs.CLcited by 4

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Multi-judge evaluation is increasingly used to assess LLMs and reward models, and the prevailing heuristic is to curate: keep the most accurate judges and discard weaker ones. We show that this heuristic can reverse when the target is not point accuracy, but calibrated probabilistic evaluation from a labeled calibration set. Holding the aggregation and calibration procedures fixed, we compare accuracy-ranked top-k judge selection with using the full judge panel. Across four labeled pairwise-evaluation benchmarks spanning LLM-as-judge and reward-model settings, the calibrated full panel consistently outperforms accuracy-based selection. On RewardBench2, retaining all judges achieves negative log-likelihood (NLL) of 0.006 versus 0.013 under top-5 selection, halving the calibration error. This advantage persists after judge-family deduplication and against stronger same-pipeline subset search. We explain this reversal with oracle analyses showing that the optimal calibrated risk under proper scoring rules cannot increase when additional judge signals are made available, and that even below-chance judges can be useful when their biases are learnable and their signals are non-redundant. The resulting operating principle is simple: in multi-judge evaluation with labeled calibration data, do not discard weak judges by accuracy alone; keep them when they are parseable, non-redundant, and calibratable.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 31 of its 46 references that are in the map. First posted on arXiv 10 May 2026.

Builds on

31 of 46 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Efficient Bayesian Inference from Noisy Pairwise Comparisons10 Oct
  2. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference7 Mar

Other references · 29

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2017On Calibration of Modern Neural Networkscited by 4,860
  4. 2019Conformal Prediction Under Covariate Shiftcited by 511
  5. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 480
  6. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 354
  7. 2022Conformal prediction beyond exchangeabilitycited by 348
  8. 2019Verified Uncertainty Calibrationcited by 307
  9. 2024JudgeBench: A Benchmark for Evaluating LLM-based Judgescited by 294
  10. 2023Evaluating Large Language Models at Evaluating Instruction Followingcited by 290
  11. 2025RewardBench 2: Advancing Reward Model Evaluationcited by 136
  12. 2024Trust or Escalate: LLM Judges with Provable Guarantees for Human Agreementcited by 99
17 more
  1. 2019Veridical Data Sciencecited by 65
  2. 2024Limits to scalable evaluation at the frontier: LLM as Judge won't beat twice the datacited by 46
  3. 2024Prediction-Powered Ranking of Large Language Modelscited by 29
  4. 2023Adaptive conformal classification with noisy labelscited by 24
  5. 2022Label Noise Robustness of Conformal Predictioncited by 19
  6. 2025Overconfidence in LLM-as-a-Judge: Diagnosis and Confidence-Driven Solutioncited by 17
  7. 2024Self-Calibrating Conformal Predictioncited by 16
  8. 2025Leveraging LLMs as Meta-Judges: A Multi-Agent Framework for Evaluating LLM Judgmentscited by 16
  9. 2024QA-Calibration of Language Model Confidence Scorescited by 13
  10. 2026A Judge-Aware Ranking Framework for Evaluating Large Language Models without Ground Truthcited by 7
  11. 2026Who can we trust? LLM-as-a-jury for Comparative Assessmentcited by 7
  12. 2025Generalized Venn and Venn-Abers Calibration with Applications in Conformal Predictioncited by 6
  13. 2025Quantitative LLM Judgescited by 6
  14. 2025CLEAR: Calibrated Learning for Epistemic and Aleatoric Riskcited by 6
  15. 2025Calibrating LLM Judges: Linear Probes for Fast and Reliable Uncertainty Estimationcited by 6
  16. 2026PRECISE: Reducing the Bias of LLM Evaluations Using Prediction-Powered Ranking Estimationcited by 6
  17. 2025Distribution-Calibrated Inference Time Compute for Thinking LLM-as-a-Judgecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026A Finite-Calibration Regime Map for LLM Judge Panelscited by 2
  2. 2026Stopping and Routing LLM Judge Panelscited by 1
  3. 2026Aggregate Disambiguation Systemscited by 0
  4. 2026RAIM: Robust Aggregation of Inexpensive Models for Hallucination Detectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026On Cost-Effective LLM-as-a-Judge Improvement Techniques
  2. 2026Judge, Retrieve, or Abstain: Uncertainty-Guarded LLM Judging with Provable Risk Guarantees
  3. 2026LLM Judge Validation Under Sparse Overlap: From Inference to Design
  4. 2026Rethinking Verbalized Confidence for LLM-as-a-Judge: A Compatibility Shift on Post-2025 Proprietary Models
  5. 2026CLM-as-a-Judge: Evaluating an Open Contrastive Decision Model on Public Judge Benchmarks
  6. 2026RUBRIC-ARROW: Alternating Pointwise Rubric Reward Modeling for LLM Post-training in Non-verifiable Domains
  7. 2026Toward Robust LLM-Based Judges: Taxonomic Bias Evaluation and Debiasing Optimization
  8. 2025When Judgment Becomes Noise: How Design Failures in LLM Judge Benchmarks Silently Undermine Validity