mlsanityData to 7 Oct 2026

Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators

arXiv 2403.16950 ↗First posted 25 Mar 2024cs.CL cs.AI cs.LGcited by 146

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated promising capabilities as automatic evaluators in assessing the quality of generated natural language. However, LLMs still exhibit biases in evaluation and often struggle to generate coherent evaluations that align with human assessments. In this work, we first conduct a systematic study of the misalignment between LLM evaluators and human evaluation, revealing that existing calibration methods aimed at mitigating biases of LLMs are insufficient for effectively aligning LLM evaluators. Inspired by the use of preference data in RLHF, we formulate the evaluation as a ranking problem and introduce Pairwise-preference Search (PAIRS), an uncertainty-guided search-based rank aggregation method that employs LLMs to conduct pairwise comparisons locally and efficiently ranks candidate texts globally. PAIRS achieves state-of-the-art performance on representative evaluation tasks in long-form generations and demonstrates significant improvements over direct scoring. Furthermore, we provide insights into the role of pairwise preference in quantifying the transitivity of LLMs and demonstrate how PAIRS benefits from calibration using debiased pairwise evaluations.

Key facts. Cited by 146 papers in the map (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 45 of its 58 references that are in the map. First posted on arXiv 25 Mar 2024.

Builds on

45 of 58 references.

Closest ancestors · 1

Same direction, newest first

  1. 2020Learning to summarize from human feedback2 Sep

Other references · 44

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  7. 2017Deep reinforcement learning from human preferencescited by 3,846
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  9. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  10. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  11. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  12. 2023PaLM 2 Technical Reportcited by 1,154
32 more
  1. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  2. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
  3. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  4. 2020SummEval: Re-evaluating Summarization Evaluationcited by 666
  5. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 630
  6. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 454
  7. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 420
  8. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 416
  9. 2023GPTScore: Evaluate as You Desirecited by 357
  10. 2023Large Language Models are Effective Text Rankers with Pairwise Ranking Promptingcited by 302
  11. 2018Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategiescited by 292
  12. 2023Evaluating Large Language Models at Evaluating Instruction Followingcited by 290
  13. 2022Towards a Unified Multi-Dimensional Evaluator for Text Generationcited by 264
  14. 2023Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questionscited by 249
  15. 2023Wider and Deeper LLM Networks are Fairer LLM Evaluatorscited by 91
  16. 2023Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineeringcited by 91
  17. 2023Verbosity Bias in Preference Labeling by Large Language Modelscited by 89
  18. 2023Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: An Empirical Studycited by 85
  19. 2022Of Human Criteria and Automatic Metrics: A Benchmark of the Evaluation of Story Generationcited by 73
  20. 2023Split and Merge: Aligning Position Biases in LLM-based Evaluatorscited by 71
  21. 2023Large Language Models are Not Yet Human-Level Evaluators for Abstractive Summarizationcited by 70
  22. 2023LLM Comparative Assessment: Zero-shot NLG Evaluation through Pairwise Comparisons using Large Language Modelscited by 66
  23. 2023Calibrating LLM-Based Evaluatorcited by 53
  24. 2023Better Zero-Shot Reasoning with Self-Adaptive Promptingcited by 52
  25. 2023AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuningcited by 49
  26. 2024PROXYQA: An Alternative Framework for Evaluating Long-Form Text Generation with Large Language Modelscited by 23
  27. 2023Generative Calibration for In-context Learningcited by 20
  28. 2023Survival of the Most Influential Prompts: Efficient Black-Box Prompt Search via Clustering and Pruningcited by 17
  29. 2023Universal Self-Adaptive Promptingcited by 14
  30. 2023Large Language Models are Miscalibrated In-Context Learnerscited by 12
  31. 2024Unlocking Structure Measuring: Introducing PDD, an Automatic Metric for Positional Discourse Coherencecited by 7
  32. 2024TOAD: Task-Oriented Automatic Dialogs with Diverse Response Stylescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 31; 2025: 72; 2026: 43.

146 papers.

Sort:

By area:

  1. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
  2. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 451
  3. 2024Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judgecited by 249
  4. 2024LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Taskscited by 236
  5. 2024Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judgescited by 197
  6. 2025Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectivescited by 120
  7. 2024Can LLM be a Personalized Judge?cited by 99
  8. 2024Agents' Room: Narrative Generation through Multi-step Collaborationcited by 57
  9. 2025No Free Labels: Limitations of LLM-as-a-Judge Without Human Groundingcited by 50
  10. 2024DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processingcited by 49
  11. 2024LongGenBench: Benchmarking Long-Form Generation in Long Context LLMscited by 47
  12. 2024Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Ratescited by 42
38 more
  1. 2025Evaluating Scoring Bias in LLM-as-a-Judgecited by 42
  2. 2025Self-Supervised Prompt Optimizationcited by 37
  3. 2025Improving LLM-as-a-Judge Inference with the Judgment Distributioncited by 36
  4. 2024Evaluating the Evaluator: Measuring LLMs' Adherence to Task Evaluation Instructionscited by 34
  5. 2025Investigating Non-Transitivity in LLM-as-a-Judgecited by 33
  6. 2024LLM-as-a-Judge & Reward Model: What They Can and Cannot Docited by 32
  7. 2024CalibraEval: Calibrating Prediction Distribution to Mitigate Selection Bias in LLMs-as-Judgescited by 32
  8. 2025VISTA: A Test-Time Self-Improving Video Generation Agentcited by 26
  9. 2024Fairer Preferences Elicit Improved Human-Aligned Large Language Model Judgmentscited by 25
  10. 2024Reference-Guided Verdict: LLMs-as-Judges in Automatic Evaluation of Free-Form QAcited by 25
  11. 2024Efficient LLM Comparative Assessment: a Product of Experts Framework for Pairwise Comparisonscited by 22
  12. 2025Does Context Matter? ContextualJudgeBench for Evaluating LLM-based Judges in Contextual Settingscited by 22
  13. 2025Brains vs. Bytes: Evaluating LLM Proficiency in Olympiad Mathematicscited by 22
  14. 2025Process-based Self-Rewarding Language Modelscited by 21
  15. 2025Sentient Agent as a Judge: Evaluating Higher-Order Social Cognition in Large Language Modelscited by 21
  16. 2025AI Alignment at Your Discretioncited by 20
  17. 2025An Empirical Analysis of Uncertainty in Large Language Model Evaluationscited by 20
  18. 2024FormalAlign: Automated Alignment Evaluation for Autoformalizationcited by 19
  19. 2026Learning to Judge: LLMs Designing and Applying Evaluation Rubricscited by 18
  20. 2025An Empirical Study of LLM-as-a-Judge: How Design Choices Impact Evaluation Reliabilitycited by 17
  21. 2024Grade Like a Human: Rethinking Automated Assessment with Large Language Modelscited by 16
  22. 2024Re-evaluating Automatic LLM System Ranking for Alignment with Human Preferencecited by 16
  23. 2025Retrieval-Based Interleaved Visual Chain-of-Thought in Real-World Driving Scenarioscited by 16
  24. 2025Ranked Voting based Self-Consistency of Large Language Modelscited by 16
  25. 2025Rankers, Judges, and Assistants: Towards Understanding the Interplay of LLMs in Information Retrieval Evaluationcited by 15
  26. 2025GenSelect: A Generative Approach to Best-of-Ncited by 14
  27. 2024Towards Acyclic Preference Evaluation of Language Models via Multiple Evaluatorscited by 13
  28. 2025C3AI: Crafting and Evaluating Constitutions for Constitutional AIcited by 13
  29. 2025Ask a Strong LLM Judge when Your Reward Model is Uncertaincited by 13
  30. 2024DnA-Eval: Enhancing Large Language Model Evaluation through Decomposition and Aggregationcited by 12
  31. 2025Multi2: Multi-Agent Test-Time Scalable Framework for Multi-Document Processingcited by 12
  32. 2024TradExpert: Revolutionizing Trading with Mixture of Expert LLMscited by 11
  33. 2025Tuning LLM Judge Design Decisions for 1/1000 of the Costcited by 11
  34. 2025What Is That Talk About? A Video-to-Text Summarization Dataset for Scientific Presentationscited by 11
  35. 2025IntentionESC: An Intention-Centered Framework for Enhancing Emotional Support in Dialogue Systemscited by 11
  36. 2025MLLM as a UI Judge: Benchmarking Multimodal LLMs for Predicting Human Perception of User Interfacescited by 11
  37. 2026Counting on Consensus: Selecting the Right Inter-annotator Agreement Metric for NLP Annotation and Evaluationcited by 11
  38. 2024Reversal of Thought: Enhancing Large Language Models with Preference-Guided Reverse Reasoning Warm-upcited by 10

50 most cited of 146 All 146 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024HD-Eval: Aligning Large Language Model Evaluators Through Hierarchical Criteria Decomposition
  2. 2024The Comparative Trap: Pairwise Comparisons Amplifies Biased Preferences of LLM Evaluators
  3. 2023Peering Through Preferences: Unraveling Feedback Acquisition for Aligning Large Language Models
  4. 2026Pair Difficulty Matters: Rethinking Pairwise LLM-as-a-Judge Evaluation and Consistency
  5. 2025Aligning Black-box Language Models with Human Judgments
  6. 2025On Evaluating LLM Alignment by Evaluating LLMs as Judges
  7. 2024Sample-Efficient Human Evaluation of Large Language Models via Maximum Discrepancy Competition
  8. 2023Exploring the Reliability of Large Language Models as Customized Evaluators for Diverse NLP Tasks