mlsanityData to 7 Oct 2026

Style Over Substance: Evaluation Biases for Large Language Models

arXiv 2307.03025 ↗First posted 6 Jul 2023cs.CLcited by 68

Not on the map. Why?

Abstract (arXiv)

As large language models (LLMs) continue to advance, accurately and comprehensively evaluating their performance becomes increasingly challenging. Ranking the relative performance of LLMs based on Elo ratings, according to human judgment, is gaining more popularity. However, the extent to which humans and LLMs are capable evaluators remains uncertain. This study investigates the behavior of crowd-sourced and expert annotators, as well as LLMs, when comparing outputs from different models. To achieve this, we curate a dataset of intentionally flawed machine-generated answers. Our findings reveal a concerning bias in the evaluation process, as answers with factual errors are rated more favorably than answers that are too short or contained grammatical errors. To address this issue, we propose independently evaluating machine-generated text across multiple dimensions, rather than merging all the evaluation aspects into a single score. We instantiate this idea with the Elo rating system, resulting in the Multi-Elo Rating System (MERS). Empirical results from our study reveal that this proposed approach significantly enhances the quality of LLM-based evaluations, particularly in terms of factual accuracy. However, there is no significant improvement in crowd-sourced-based evaluations, indicating the need for further investigation.

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). Builds on 36 of its 51 references that are in the map. First posted on arXiv 6 Jul 2023.

Builds on

36 of 51 references.

References · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  10. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  11. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  12. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
24 more
  1. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  2. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  3. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  4. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  5. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  6. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  7. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  8. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  9. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  10. 2022Galactica: A Large Language Model for Sciencecited by 736
  11. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  12. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 436
  13. 2023PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimizationcited by 311
  14. 2021All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Textcited by 287
  15. 2023The False Promise of Imitating Proprietary LLMscited by 254
  16. 2023Macaw-LLM: Multi-Modal Language Modeling with Image, Audio, Video, and Text Integrationcited by 168
  17. 2022Fine-tuned Language Models are Continual Learnerscited by 152
  18. 2021Automated Fact-Checking for Assisting Human Fact-Checkerscited by 144
  19. 2021PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computationcited by 134
  20. 2023LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructionscited by 124
  21. 2023Bactrian-X: Multilingual Replicable Instruction-Following Models with Low-Rank Adaptationcited by 79
  22. 2020Learning from Task Descriptionscited by 78
  23. 2022InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuningcited by 39
  24. 2022ConTinTin: Continual Learning from Task Instructionscited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 27; 2025: 18; 2026: 17.

68 papers.

Sort:

By area:

  1. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 865
  2. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 354
  3. 2024Humans or LLMs as the Judge? A Study on Judgement Biasescited by 314
  4. 2024Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judgescited by 197
  5. 2023OctoPack: Instruction Tuning Code Large Language Modelscited by 178
  6. 2024Lessons from the Trenches on Reproducible Evaluation of Language Modelscited by 172
  7. 2023Orca 2: Teaching Small Language Models How to Reasoncited by 157
  8. 2023Benchmarking Cognitive Biases in Large Language Models as Evaluatorscited by 147
  9. 2024Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Eracited by 97
  10. 2024VL-RewardBench: A Challenging Benchmark for Vision-Language Generative Reward Modelscited by 85
  11. 2024LLM-based NLG Evaluation: Current Status and Challengescited by 71
  12. 2024Systematic Evaluation of LLM-as-a-Judge in LLM Alignment Tasks: Explainable Metrics and Diverse Prompt Templatescited by 71
38 more
  1. 2023Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Modelscited by 68
  2. 2024(Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Textscited by 49
  3. 2024Are We There Yet? Revealing the Risks of Utilizing Large Language Models in Scholarly Peer Reviewcited by 49
  4. 2024Inverse Constitutional AI: Compressing Preferences into Principlescited by 39
  5. 2024Style Outweighs Substance: Failure Modes of LLM Judges in Alignment Benchmarkingcited by 35
  6. 2023Aligning Large Language Models with Human: A Surveycited by 31
  7. 2025Search Arena: Analyzing Search-Augmented LLMscited by 30
  8. 2023MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteriacited by 27
  9. 2025Pairwise or Pointwise? Evaluating Feedback Protocols for Bias in LLM-Based Evaluationcited by 23
  10. 2025AI Alignment at Your Discretioncited by 20
  11. 2025Multi-Agent Debate for LLM Judges with Adaptive Stability Detectioncited by 20
  12. 2024RevisEval: Improving LLM-as-a-Judge via Response-Adapted Referencescited by 19
  13. 2024Eureka: Evaluating and Understanding Large Foundation Modelscited by 18
  14. 2024On Catastrophic Inheritance of Large Foundation Modelscited by 15
  15. 2025LikePhys: Evaluating Intuitive Physics Understanding in Video Diffusion Models via Likelihood Preferencecited by 14
  16. 2025Flattery, Fluff, and Fog: Diagnosing and Mitigating Idiosyncratic Biases in Preference Modelscited by 13
  17. 2024The Responsible Foundation Model Development Cheatsheet: A Review of Tools & Resourcescited by 11
  18. 2024Concept -- An Evaluation Protocol on Conversational Recommender Systems with System-centric and User-centric Factorscited by 9
  19. 2025STREAM (ChemBio): A Standard for Transparently Reporting Evaluations in AI Model Reportscited by 9
  20. 2024MONAL: Model Autophagy Analysis for Modeling Human-AI Interactionscited by 7
  21. 2025Multi-Agent LLM Judge: automatic personalized LLM judge design for evaluating natural language generation applicationscited by 7
  22. 2025Inverse Reinforcement Learning Meets Large Language Model Post-Training: Basics, Advances, and Opportunitiescited by 7
  23. 2026Judging the Judges: A Systematic Evaluation of Bias Mitigation Strategies in LLM-as-a-Judge Pipelinescited by 7
  24. 2024LLM See, LLM Do: Guiding Data Generation to Target Non-Differentiable Objectivescited by 6
  25. 2025Skewed Score: A statistical framework to assess autograderscited by 6
  26. 2025Who's Your Judge? On the Detectability of LLM-Generated Judgmentscited by 6
  27. 2024SeRA: Self-Reviewing and Alignment of Large Language Models using Implicit Reward Marginscited by 5
  28. 2024Towards Effective Discrimination Testing for Generative AIcited by 5
  29. 2025IberBench: LLM Evaluation on Iberian Languagescited by 5
  30. 2025Datasheets Aren't Enough: DataRubrics for Automated Quality Metrics and Accountabilitycited by 5
  31. 2024Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluationcited by 3
  32. 2024EQUATOR: A Deterministic Framework for Evaluating LLM Reasoning with Open-Ended Questions. # v1.0.0-betacited by 3
  33. 2024Fine-tuning and Prompt Engineering with Cognitive Knowledge Graphs for Scholarly Knowledge Organizationcited by 2
  34. 2025Word Overuse and Alignment in Large Language Models: The Influence of Learning from Human Feedbackcited by 2
  35. 2026Designing Reliable LLM-Assisted Rubric Scoring for Constructed Responses: Evidence from Physics Examscited by 2
  36. 2026Scaffold, Not Vocabulary? A Controlled, Two-Tier, Pre-Registered Study of a Popperian Code-Generation Skillcited by 2
  37. 2026Don't Claim Benchmark-Oriented Optimization Improves General Coding Capability -- Diverse Evaluation Is Requiredcited by 2
  38. 2024Impact of Model Size on Fine-tuned LLM Performance in Data-to-Text Generation: A State-of-the-Art Investigationcited by 1

50 most cited of 68 All 68 (JSON)