mlsanityData to 14 Oct 2026

Fairer Preferences Elicit Improved Human-Aligned Large Language Model Judgments

arXiv 2406.11370 ↗First posted 17 Jun 2024cs.CL cs.AI cs.CYcited by 25

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Large language models (LLMs) have shown promising abilities as cost-effective and reference-free evaluators for assessing language generation quality. In particular, pairwise LLM evaluators, which compare two generated texts and determine the preferred one, have been employed in a wide range of applications. However, LLMs exhibit preference biases and worrying sensitivity to prompt designs. In this work, we first reveal that the predictive preference of LLMs can be highly brittle and skewed, even with semantically equivalent instructions. We find that fairer predictive preferences from LLMs consistently lead to judgments that are better aligned with humans. Motivated by this phenomenon, we propose an automatic Zero-shot Evaluation-oriented Prompt Optimization framework, ZEPO, which aims to produce fairer preference decisions and improve the alignment of LLM evaluators with human judgments. To this end, we propose a zero-shot learning objective based on the preference decision fairness. ZEPO demonstrates substantial performance improvements over state-of-the-art LLM evaluators, without requiring labeled data, on representative meta-evaluation benchmarks. Our findings underscore the critical correlation between preference fairness and human alignment, positioning ZEPO as an efficient prompt optimizer for bridging the gap between LLM evaluators and human judgments.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 31 of its 46 references that are in the map. First posted on arXiv 17 Jun 2024.

Builds on

31 of 46 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024LLM Evaluators Recognize and Favor Their Own Generations15 Apr
  2. 2024Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators25 Mar

Other references · 29

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  6. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  7. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  8. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  9. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  10. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  11. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  12. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
17 more
  1. 2020SummEval: Re-evaluating Summarization Evaluationcited by 674
  2. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
  3. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 486
  4. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 366
  5. 2023Large Language Models are Effective Text Rankers with Pairwise Ranking Promptingcited by 307
  6. 2023Evaluating Large Language Models at Evaluating Instruction Followingcited by 302
  7. 2018Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategiescited by 294
  8. 2022Towards a Unified Multi-Dimensional Evaluator for Text Generationcited by 264
  9. 2023Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questionscited by 263
  10. 2023PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimizationcited by 233
  11. 2020USR: An Unsupervised and Reference Free Evaluation Metric for Dialog Generationcited by 188
  12. 2023Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineeringcited by 94
  13. 2023Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: An Empirical Studycited by 85
  14. 2023LLM Comparative Assessment: Zero-shot NLG Evaluation through Pairwise Comparisons using Large Language Modelscited by 71
  15. 2023Better Zero-Shot Reasoning with Self-Adaptive Promptingcited by 52
  16. 2023Self-ICL: Zero-Shot In-Context Learning with Self-Generated Demonstrationscited by 42
  17. 2023Survival of the Most Influential Prompts: Efficient Black-Box Prompt Search via Clustering and Pruningcited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 15; 2026: 6.

25 papers.

Sort:

By area:

  1. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 494
  2. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 465
  3. 2025Multi-Agent Design: Optimizing Agents with Better Prompts and Topologiescited by 113
  4. 2025No Free Labels: Limitations of LLM-as-a-Judge Without Human Groundingcited by 54
  5. 2025Self-Supervised Prompt Optimizationcited by 37
  6. 2025Investigating Non-Transitivity in LLM-as-a-Judgecited by 34
  7. 2025Analyzing Uncertainty of LLM-as-a-Judge: Interval Evaluations with Conformal Predictioncited by 27
  8. 2025reWordBench: Benchmarking and Improving the Robustness of Reward Models with Transformed Inputscited by 25
  9. 2025From Few to Many: Self-Improving Many-Shot Reasoners Through Iterative Optimization and Generationcited by 21
  10. 2025An Empirical Analysis of Uncertainty in Large Language Model Evaluationscited by 20
  11. 2025Maestro: Self-Improving Text-to-Image Generation via Agent Orchestrationcited by 19
  12. 2024Aligning with Logic: Measuring, Evaluating and Improving Logical Preference Consistency in Large Language Modelscited by 12
13 more
  1. 2025Tuning LLM Judge Design Decisions for 1/1000 of the Costcited by 11
  2. 2025Improving Preference Extraction In LLMs By Identifying Latent Knowledge Through Classifying Probescited by 9
  3. 2024Understanding Likelihood Over-optimisation in Direct Alignment Algorithmscited by 7
  4. 2026From Rubrics to Reliable Scores: Evidence-Grounded Text Evaluation with LLM Judgescited by 7
  5. 2025MDSEval: A Meta-Evaluation Benchmark for Multimodal Dialogue Summarizationcited by 3
  6. 2025Agentic Policy Optimization via Instruction-Policy Co-Evolutioncited by 3
  7. 2025SynCast: Synergizing Contradictions in Precipitation Nowcasting via Diffusion Sequential Preference Optimizationcited by 2
  8. 2025HypoEval: Hypothesis-Guided Evaluation for Natural Language Generationcited by 1
  9. 2026UPA: Unsupervised Prompt Agent via Tree-Based Search and Selectioncited by 1
  10. 2026SPEAR: Code-Augmented Agentic Prompt Optimizationcited by 1
  11. 2026Agreement Overstates Evidence: Error Dependence in LLM Judge Consensuscited by 1
  12. 2026COGNAC at SemEval-2026 Task 5: LLM Ensembles for Human-Level Word Sense Plausibility Rating in Challenging Narrativescited by 0
  13. 2026Multiple latent orderings better predict language model preferencescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Peering Through Preferences: Unraveling Feedback Acquisition for Aligning Large Language Models
  2. 2023On Diversified Preferences of Large Language Model Alignment
  3. 2024Comparing Bad Apples to Good Oranges: Aligning Large Language Models via Joint Preference Optimization
  4. 2024HD-Eval: Aligning Large Language Model Evaluators Through Hierarchical Criteria Decomposition
  5. 2024A Survey on Human Preference Learning for Large Language Models
  6. 2024Towards a Unified View of Preference Learning for Large Language Models: A Survey
  7. 2024Dissecting Human and LLM Preferences
  8. 2025On Evaluating LLM Alignment by Evaluating LLMs as Judges