mlsanityData to 14 Oct 2026

Benchmarking Cognitive Biases in Large Language Models as Evaluators

arXiv 2309.17012 ↗First posted 29 Sep 2023cs.CL cs.AI cs.LGcited by 150

Not on the map. Why?

Abstract (arXiv)

Large Language Models are cognitively biased judges. Large Language Models (LLMs) have recently been shown to be effective as automatic evaluators with simple prompting and in-context learning. In this work, we assemble 15 LLMs of four different size ranges and evaluate their output responses by preference ranking from the other LLMs as evaluators, such as System Star is better than System Square. We then evaluate the quality of ranking outputs introducing the Cognitive Bias Benchmark for LLMs as Evaluators (CoBBLEr), a benchmark to measure six different cognitive biases in LLM evaluation outputs, such as the Egocentric bias where a model prefers to rank its own outputs highly in evaluation. We find that LLMs are biased text quality evaluators, exhibiting strong indications on our bias benchmark (average of 40% of comparisons across all models) within each of their evaluations that question their robustness as evaluators. Furthermore, we examine the correlation between human and machine preferences and calculate the average Rank-Biased Overlap (RBO) score to be 49.6%, indicating that machine preferences are misaligned with humans. According to our findings, LLMs may still be unable to be utilized for automatic annotation aligned with human preferences. Our project page is at: https://minnesotanlp.github.io/cobbler.

Key facts. Cited by 150 papers in the map (data through 14 Oct 2026). Builds on 36 of its 76 references that are in the map. First posted on arXiv 29 Sep 2023.

Builds on

36 of 76 references.

References · 36

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  7. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  8. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  9. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  10. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  11. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  12. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
24 more
  1. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  2. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  3. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  4. 2023Large Language Modelscited by 780
  5. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  6. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
  7. 2019ELI5: Long Form Question Answeringcited by 575
  8. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 422
  9. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 420
  10. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 360
  11. 2023Benchmarking Foundation Models with Language-Model-as-an-Examinercited by 172
  12. 2023LM vs LM: Detecting Factual Errors via Cross Examinationcited by 156
  13. 2023Human-like Summarization Evaluation with ChatGPTcited by 126
  14. 2023PRD: Peer Rank and Discussion Improve Large Language Model based Evaluationscited by 114
  15. 2022Capturing Failures of Large Language Models via Human Cognitive Biasescited by 98
  16. 2022PEER: A Collaborative Language Modelcited by 86
  17. 2023TrueTeacher: Learning Factual Consistency Evaluation with Large Language Modelscited by 83
  18. 2023ChatGPT as a Factual Inconsistency Evaluator for Text Summarizationcited by 75
  19. 2023Style Over Substance: Evaluation Biases for Large Language Modelscited by 70
  20. 2023CoEdIT: Text Editing by Task-Specific Instruction Tuningcited by 68
  21. 2019Earlier Isn't Always Better: Sub-aspect Analysis on Corpus and System Biases in Summarizationcited by 28
  22. 2021Does BERT Learn as Humans Perceive? Understanding Linguistic Styles through Lexicacited by 23
  23. 2023Challenging the appearance of machine intelligence: Cognitive bias in LLMs and Best Practices for Adoptioncited by 21
  24. 2022Rank List Sensitivity of Recommender Systems to Interaction Perturbationscited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 42; 2025: 53; 2026: 53.

150 papers.

Sort:

By area:

  1. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 897
  2. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  3. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 494
  4. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 465
  5. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 368
  6. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 360
  7. 2025Preference Leakage: A Contamination Problem in LLM-as-a-judgecited by 125
  8. 2024Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinementcited by 114
  9. 2024Trust or Escalate: LLM Judges with Provable Guarantees for Human Agreementcited by 105
  10. 2023SeaLLMs -- Large Language Models for Southeast Asiacited by 104
  11. 2024Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Eracited by 98
  12. 2024A Survey on Multilingual Large Language Models: Corpora, Alignment, and Biascited by 84
38 more
  1. 2024Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluationcited by 74
  2. 2024SafeBench: A Safety Evaluation Framework for Multimodal Large Language Modelscited by 69
  3. 2024Active Preference Learning for Large Language Modelscited by 58
  4. 2025No Free Labels: Limitations of LLM-as-a-Judge Without Human Groundingcited by 54
  5. 2024Limits to scalable evaluation at the frontier: LLM as Judge won't beat twice the datacited by 51
  6. 2025LLMs for Explainable AI: A Comprehensive Surveycited by 45
  7. 2023Instruction-tuning Aligns LLMs to the Human Braincited by 40
  8. 2025Neither Valid nor Reliable? Investigating the Use of LLMs as Judgescited by 39
  9. 2025Do LLM Evaluators Prefer Themselves for a Reason?cited by 38
  10. 2024Anchoring Bias in Large Language Models: An Experimental Studycited by 37
  11. 2024Tuning Large Multimodal Models for Videos using Reinforcement Learning from AI Feedbackcited by 36
  12. 2025Play Favorites: A Statistical Method to Measure Self-Bias in LLM-as-a-Judgecited by 36
  13. 2024Direct Judgement Preference Optimizationcited by 33
  14. 2025JarvisEvo: Towards a Self-Evolving Photo Editing Agent with Synergistic Editor-Evaluator Optimizationcited by 31
  15. 2025Language Models Fail to Introspect About Their Knowledge of Languagecited by 30
  16. 2025Assessing Judging Bias in Large Reasoning Models: An Empirical Studycited by 30
  17. 2025How to Correctly Report LLM-as-a-Judge Evaluationscited by 30
  18. 2026GameDevBench: Evaluating Agentic Capabilities Through Game Developmentcited by 28
  19. 2024Under the Surface: Tracking the Artifactuality of LLM-Generated Datacited by 24
  20. 2025Pairwise or Pointwise? Evaluating Feedback Protocols for Bias in LLM-Based Evaluationcited by 24
  21. 2025Shrinking the Generation-Verification Gap with Weak Verifierscited by 24
  22. 2024Emoji Attack: Enhancing Jailbreak Attacks Against Judge LLM Detectioncited by 23
  23. 2025Benchmarking Adversarial Robustness to Bias Elicitation in Large Language Models: Scalable Automated Assessment with LLM-as-a-Judgecited by 23
  24. 2025Multi-Agent Debate for LLM Judges with Adaptive Stability Detectioncited by 23
  25. 2024From Calculation to Adjudication: Examining LLM judges on Mathematical Reasoning Taskscited by 22
  26. 2026Large Language Model Reasoning Failurescited by 21
  27. 2024Cognitive Biases in Large Language Models: A Survey and Mitigation Experimentscited by 20
  28. 2025An Empirical Analysis of Uncertainty in Large Language Model Evaluationscited by 20
  29. 2025VeriPlan: Integrating Formal Verification and LLMs into End-User Planningcited by 20
  30. 2025Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Modelscited by 20
  31. 2024Large Language Model as an Assignment Evaluator: Insights, Feedback, and Challenges in a 1000+ Student Coursecited by 19
  32. 2025Position is Power: System Prompts as a Mechanism of Bias in Large Language Models (LLMs)cited by 19
  33. 2026Learning to Judge: LLMs Designing and Applying Evaluation Rubricscited by 19
  34. 2025Risk Analysis Techniques for Governed LLM-based Multi-Agent Systemscited by 18
  35. 2024AcTracer: Active Testing of Large Language Model via Multi-Stage Samplingcited by 17
  36. 2024PM-LLM-Benchmark: Evaluating Large Language Models on Process Mining Taskscited by 16
  37. 2024Training Language Models to Win Debates with Self-Play Improves Judge Accuracycited by 16
  38. 2026Tiny Aya: Bridging Scale and Multilingual Depthcited by 15

50 most cited of 150 All 150 (JSON)