mlsanityData to 7 Oct 2026

A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations

arXiv 2407.04069 ↗First posted 4 Jul 2024cs.CL cs.AI cs.LGcited by 84

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have recently gained significant attention due to their remarkable capabilities in performing diverse tasks across various domains. However, a thorough evaluation of these models is crucial before deploying them in real-world applications to ensure they produce reliable performance. Despite the well-established importance of evaluating LLMs in the community, the complexity of the evaluation process has led to varied evaluation setups, causing inconsistencies in findings and interpretations. To address this, we systematically review the primary challenges and limitations causing these inconsistencies and unreliable evaluations in various steps of LLM evaluation. Based on our critical review, we present our perspectives and recommendations to ensure LLM evaluations are reproducible, reliable, and robust.

Key facts. Cited by 84 papers in the map (data through 7 Oct 2026). Builds on 134 of its 185 references that are in the map. First posted on arXiv 4 Jul 2024.

Builds on

134 of 185 references.

References · 134

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  8. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  9. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  10. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  11. 2021Program Synthesis with Large Language Modelscited by 3,509
  12. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
38 more
  1. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  2. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  3. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  4. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  5. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  6. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  7. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  8. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  9. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  10. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  11. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  12. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  13. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  14. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  15. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  16. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  17. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  18. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  19. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  20. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  21. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  22. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  23. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  24. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
  25. 2022Red Teaming Language Models with Language Modelscited by 983
  26. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 929
  27. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  28. 2021Measuring Coding Challenge Competence With APPScited by 879
  29. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  30. 2016A Diagram Is Worth A Dozen Imagescited by 793
  31. 2023StarCoder: may the source be with you!cited by 790
  32. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
  33. 2023Large Language Modelscited by 774
  34. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
  35. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  36. 2020Aligning AI With Shared Human Valuescited by 706
  37. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 668
  38. 2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 530

50 most cited of 134 All 134 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 51; 2026: 27.

84 papers.

Sort:

By area:

  1. 2025PPTAgent: Generating and Evaluating Presentations Beyond Text-to-Slidescited by 83
  2. 2024CFBench: A Comprehensive Constraints-Following Benchmark for LLMscited by 65
  3. 2025Which of These Best Describes Multiple Choice Evaluation with LLMs? A) Forced B) Flawed C) Fixable D) All of the Abovecited by 51
  4. 2025CODESIM: Multi-Agent Code Generation and Problem Solving through Simulation-Driven Planning and Debuggingcited by 31
  5. 2024On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMscited by 24
  6. 2025Normative Evaluation of Large Language Models with Everyday Moral Dilemmascited by 23
  7. 2024Dynamic Multimodal Evaluation with Flexible Complexity by Vision-Language Bootstrappingcited by 22
  8. 2025When Can We Trust LLMs in Mental Health? Large-Scale Benchmarks for Reliable LLM Evaluationcited by 19
  9. 2025Evaluating LLM Metrics Through Real-World Capabilitiescited by 17
  10. 2025Probing the Critical Point (CritPt) of AI Reasoning: a Frontier Physics Research Benchmarkcited by 16
  11. 2025Position: Editing Large Language Models Poses Serious Safety Riskscited by 15
  12. 2025How Far Are AI Scientists from Changing the World?cited by 15
38 more
  1. 2025Improving Automatic Evaluation of Large Language Models (LLMs) in Biomedical Relation Extraction via LLMs-as-the-Judgecited by 13
  2. 2025Firm or Fickle? Evaluating Large Language Models Consistency in Sequential Interactionscited by 12
  3. 2025Evolutionary Perspectives on the Evaluation of LLM-Based AI Agents: A Comprehensive Surveycited by 12
  4. 2025You Don't Know Until You Click:Automated GUI Testing for Production-Ready Software Evaluationcited by 11
  5. 2026Evaluating LLMs in Finance Requires Explicit Bias Considerationcited by 11
  6. 2026Counting on Consensus: Selecting the Right Inter-annotator Agreement Metric for NLP Annotation and Evaluationcited by 11
  7. 2026GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation Taskscited by 11
  8. 2025EvaLearn: Quantifying the Learning Capability and Efficiency of LLMs via Sequential Problem Solvingcited by 10
  9. 2025What do Large Language Models Say About Animals? Investigating Risks of Animal Harm in Generated Textcited by 9
  10. 2025A validity-guided workflow for robust large language model research in psychologycited by 8
  11. 2025Evaluation of LLMs for mathematical problem solvingcited by 7
  12. 2024ReflectDiffu:Reflect between Emotion-intent Contagion and Mimicry for Empathetic Response Generation via a RL-Diffusion Frameworkcited by 6
  13. 2025Evaluating Polish linguistic and cultural competency in large language modelscited by 6
  14. 2025Understanding the Anchoring Effect of LLM with Synthetic Data: Existence, Mechanism, and Potential Mitigationscited by 6
  15. 2025AECBench: A Hierarchical Benchmark for Knowledge Evaluation of Large Language Models in the AEC Fieldcited by 6
  16. 2025ZeroSumEval: An Extensible Framework For Scaling LLM Evaluation with Inter-Model Competitioncited by 5
  17. 2025ZeroSumEval: Scaling LLM Evaluation with Inter-Model Competitioncited by 5
  18. 2025Benchmarking the Pedagogical Knowledge of Large Language Modelscited by 5
  19. 2026Stop Taking Tokenizers for Granted: They Are Core Design Decisions in Large Language Modelscited by 5
  20. 2025How to Enable Effective Cooperation Between Humans and NLP Models: A Survey of Principles, Formalizations, and Beyondcited by 4
  21. 2025Economic Evaluation of LLMscited by 4
  22. 2025AI Knowledge Assist: An Automated Approach for the Creation of Knowledge Bases for Conversational AI Agentscited by 4
  23. 2025EvalCards: A Framework for Standardized Evaluation Reportingcited by 4
  24. 2026From Text to Voice: A Reproducible and Verifiable Framework for Evaluating Tool Calling LLM Agentscited by 4
  25. 2024TaskEval: Assessing Difficulty of Code Generation Tasks for Large Language Modelscited by 3
  26. 2025Position: Beyond Assistance -- Reimagining LLMs as Ethical and Adaptive Co-Creators in Mental Health Carecited by 3
  27. 2026Beyond Reproducibility: Token Probabilities Expose Large Language Model Nondeterminismcited by 3
  28. 2024ExpressivityBench: Can LLMs Communicate Implicitly?cited by 2
  29. 2025From Stability to Inconsistency: A Study of Moral Preferences in LLMscited by 2
  30. 2025Tool-Aided Evolutionary LLM for Generative Policy Toward Efficient Resource Management in Wireless Federated Learningcited by 2
  31. 2025Evaluating the Effectiveness of Cost-Efficient Large Language Models in Benchmark Biomedical Taskscited by 2
  32. 2025LLMEval-Fair: A Large-Scale Longitudinal Study on Robust and Fair Evaluation of Large Language Modelscited by 2
  33. 2025Do LLMs Feel? Teaching Emotion Recognition with Prompts, Retrieval, and Curriculum Learningcited by 2
  34. 2025ARISE: Agentic Rubric-Guided Iterative Survey Engine for Automated Scholarly Paper Generationcited by 2
  35. 2026Aligning Text, Code, and Vision: A Multi-Objective Reinforcement Learning Framework for Text-to-Visualizationcited by 2
  36. 2026Alignment Verifiability in Large Language Models: Normative Indistinguishability under Behavioral Evaluationcited by 2
  37. 2026JudgeArena: A Unified Framework for Reproducible LLM-Judge Evaluationcited by 2
  38. 2025The Imitation Game for Educational AIcited by 1

50 most cited of 84 All 84 (JSON)