# CalibraEval: Calibrating Prediction Distribution to Mitigate Selection Bias in LLMs-as-Judges

arXiv: https://arxiv.org/abs/2410.15393 · HTML: https://mlsanity.com/paper/2410.15393/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 42 references that are in the map.
- First posted on arXiv 20 Oct 2024.

## Builds on (most cited first, 26 of 26)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate](https://mlsanity.com/paper/2308.07201/) (2023)
- [BARTScore: Evaluating Generated Text as Text Generation](https://mlsanity.com/paper/2106.11520/) (2021)
- [Prometheus: Inducing Fine-grained Evaluation Capability in Language Models](https://mlsanity.com/paper/2310.08491/) (2023)
- [Large Language Models Are Not Robust Multiple Choice Selectors](https://mlsanity.com/paper/2309.03882/) (2023)
- [Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models](https://mlsanity.com/paper/2405.01535/) (2024)
- [Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge](https://mlsanity.com/paper/2410.02736/) (2024)
- [JudgeLM: Fine-tuned Large Language Models are Scalable Judges](https://mlsanity.com/paper/2310.17631/) (2023)
- [Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions](https://mlsanity.com/paper/2308.11483/) (2023)
- [Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators](https://mlsanity.com/paper/2403.16950/) (2024)
- [Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessment](https://mlsanity.com/paper/2402.14016/) (2024)
- [PRD: Peer Rank and Discussion Improve Large Language Model based Evaluations](https://mlsanity.com/paper/2307.02762/) (2023)
- [Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineering](https://mlsanity.com/paper/2309.17249/) (2023)
- [Mitigating Label Biases for In-context Learning](https://mlsanity.com/paper/2305.19148/) (2023)
- [T2Ranking: A large-scale Chinese Benchmark for Passage Ranking](https://mlsanity.com/paper/2304.03679/) (2023)
- [LeCaRDv2: A Large-Scale Chinese Legal Case Retrieval Dataset](https://mlsanity.com/paper/2310.17609/) (2023)
- [PRE: A Peer Review Based Large Language Model Evaluator](https://mlsanity.com/paper/2401.15641/) (2024)
- [Mitigating the Bias of Large Language Model Evaluation](https://mlsanity.com/paper/2409.16788/) (2024)
- [BLADE: Enhancing Black-box Large Language Models with Small Domain-Specific Models](https://mlsanity.com/paper/2403.18365/) (2024)
- [Evaluation Ethics of LLMs in Legal Domain](https://mlsanity.com/paper/2403.11152/) (2024)
- [Unsupervised Large Language Model Alignment for Information Retrieval via Contrastive Feedback](https://mlsanity.com/paper/2309.17078/) (2023)

## Built on by (most cited first, 30 of 34)

- [From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge](https://mlsanity.com/paper/2411.16594/) (2024)
- [LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods](https://mlsanity.com/paper/2412.05579/) (2024)
- [LegalAgentBench: Evaluating LLM Agents in Legal Domain](https://mlsanity.com/paper/2412.17259/) (2024)
- [Neither Valid nor Reliable? Investigating the Use of LLMs as Judges](https://mlsanity.com/paper/2508.18076/) (2025)
- [Analyzing Uncertainty of LLM-as-a-Judge: Interval Evaluations with Conformal Prediction](https://mlsanity.com/paper/2509.18658/) (2025)
- [LexRAG: Benchmarking Retrieval-Augmented Generation in Multi-Turn Legal Consultation Conversation](https://mlsanity.com/paper/2502.20640/) (2025)
- [CARE: Confounder-Aware Aggregation for Reliable LLM Evaluation](https://mlsanity.com/paper/2603.00039/) (2026)
- [Quantifying and Mitigating Self-Preference Bias of LLM Judges](https://mlsanity.com/paper/2604.22891/) (2026)
- [A Survey on Efficient Large Language Model Training: From Data-centric Perspectives](https://mlsanity.com/paper/2510.25817/) (2025)
- [CaseGen: A Benchmark for Multi-Stage Legal Case Documents Generation](https://mlsanity.com/paper/2502.17943/) (2025)
- [Efficient Inference for Noisy LLM-as-a-Judge Evaluation](https://mlsanity.com/paper/2601.05420/) (2026)
- [RankAlign: A Ranking View of the Generator-Validator Gap in Large Language Models](https://mlsanity.com/paper/2504.11381/) (2025)
- [LLM-guided Hierarchical Search for End-to-end Reasoning Intensive Retrieval](https://mlsanity.com/paper/2510.13217/) (2025)
- [Causal Judge Evaluation: Calibrated Surrogate Metrics for LLM Systems](https://mlsanity.com/paper/2512.11150/) (2025)
- [ScopeJudge: Cost-Aware Pre-Execution Gating for Offensive Security Agents](https://mlsanity.com/paper/2607.07774/) (2026)
- [SCOPE: Stochastic and Counterbiased Option Placement for Evaluating Large Language Models](https://mlsanity.com/paper/2507.18182/) (2025)
- [Quantifying and Mitigating Selection Bias in LLMs: A Transferable LoRA Fine-Tuning and Efficient Majority Voting Approach](https://mlsanity.com/paper/2511.21709/) (2025)
- [Collective Alignment in LLM Multi-Agent Systems: Disentangling Bias from Cooperation via Statistical Physics](https://mlsanity.com/paper/2605.10528/) (2026)
- [Overview of the NTCIR-18 Automatic Evaluation of LLMs (AEOLLM) Task](https://mlsanity.com/paper/2503.13038/) (2025)
- [HypoEval: Hypothesis-Guided Evaluation for Natural Language Generation](https://mlsanity.com/paper/2504.07174/) (2025)
- [DeKeyNLU: Enhancing Natural Language to SQL Generation through Task Decomposition and Keyword Extraction](https://mlsanity.com/paper/2509.14507/) (2025)
- [Are LLMs Positionally Consistent Ordinal Classifiers? A Systematic Evaluation](https://mlsanity.com/paper/2608.08869/) (2026)
- [ScopeBench: Do Agents Preserve Engagement Boundaries Under Goal Pressure?](https://mlsanity.com/paper/2609.30325/) (2026)
- [Metric assessment protocol in the context of answer fluctuation on MCQ tasks](https://mlsanity.com/paper/2507.15581/) (2025)
- [CURV: Enhancing Chart Understanding Through Curriculum Visual Grounded Reasoning](https://mlsanity.com/paper/2608.02833/) (2026)
- [Mitigating Rubric Interference in LLM Judges via On-Policy Self-Distillation](https://mlsanity.com/paper/2608.14684/) (2026)
- [Rank Reversal in Multilingual LLM Judges: A Label-Free Double-Centering Calibrator](https://mlsanity.com/paper/2608.22432/) (2026)
- [Equal Ranking Quality, Different Decisions: Measuring and Reducing Order Dependence in LLM Scorers](https://mlsanity.com/paper/2608.26762/) (2026)
- [Wrong Prediction, Right Answer: Recovering Evidence from Collapsed LLM Sequence Scores](https://mlsanity.com/paper/2608.31068/) (2026)
- [LLJ Cards: Best practices for the Use of LLMs as Judges](https://mlsanity.com/paper/2609.24516/) (2026)

## Abstract (arXiv)

The use of large language models (LLMs) as automated evaluation tools to assess the quality of generated natural language, known as LLMs-as-Judges, has demonstrated promising capabilities and is rapidly gaining widespread attention. However, when applied to pairwise comparisons of candidate responses, LLM-based evaluators often exhibit selection bias. Specifically, their judgments may become inconsistent when the option positions or ID tokens are swapped, compromising the effectiveness and fairness of the evaluation result. To address this challenge, we introduce CalibraEval, a novel label-free method for mitigating selection bias during inference. Specifically, CalibraEval reformulates debiasing as an optimization task aimed at adjusting observed prediction distributions to align with unbiased prediction distributions. To solve this optimization problem, we propose a non-parametric order-preserving algorithm (NOA). This algorithm leverages the partial order relationships between model prediction distributions, thereby eliminating the need for explicit labels and precise mathematical function modeling.Empirical evaluations of LLMs in multiple representative benchmarks demonstrate that CalibraEval effectively mitigates selection bias and improves performance compared to existing debiasing methods. This work marks a step toward building more robust and unbiased automated evaluation frameworks, paving the way for improved reliability in AI-driven assessments