mlsanityData to 7 Oct 2026

GPTEval: A Survey on Assessments of ChatGPT and GPT-4

arXiv 2308.12488 ↗First posted 24 Aug 2023cs.AI cs.CLcited by 73

Not on the map. Why?

Abstract (arXiv)

The emergence of ChatGPT has generated much speculation in the press about its potential to disrupt social and economic systems. Its astonishing language ability has aroused strong curiosity among scholars about its performance in different domains. There have been many studies evaluating the ability of ChatGPT and GPT-4 in different tasks and disciplines. However, a comprehensive review summarizing the collective assessment findings is lacking. The objective of this survey is to thoroughly analyze prior assessments of ChatGPT and GPT-4, focusing on its language and reasoning abilities, scientific knowledge, and ethical considerations. Furthermore, an examination of the existing evaluation methods is conducted, offering several recommendations for future research in evaluating large language models.

Key facts. Cited by 73 papers in the map (data through 7 Oct 2026). Builds on 69 of its 140 references that are in the map. First posted on arXiv 24 Aug 2023.

Builds on

69 of 140 references.

References · 69

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  8. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  9. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  10. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  11. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  12. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
38 more
  1. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  2. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 849
  3. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  4. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  5. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  6. 2020SummEval: Re-evaluating Summarization Evaluationcited by 666
  7. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 486
  8. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 420
  9. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 416
  10. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 356
  11. 2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causalitycited by 348
  12. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
  13. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 343
  14. 2018Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategiescited by 292
  15. 2022Towards a Unified Multi-Dimensional Evaluator for Text Generationcited by 264
  16. 2023Mathematical Capabilities of ChatGPTcited by 263
  17. 2023The False Promise of Imitating Proprietary LLMscited by 254
  18. 2023ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPTcited by 250
  19. 2023On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspectivecited by 208
  20. 2023ChatGPT: Jack of all trades, master of nonecited by 204
  21. 2023ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learningcited by 203
  22. 2023Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4cited by 202
  23. 2020USR: An Unsupervised and Reference Free Evaluation Metric for Dialog Generationcited by 186
  24. 2023Is ChatGPT A Good Translator? Yes With GPT-4 As The Enginecited by 172
  25. 2023A Categorical Archive of ChatGPT Failurescited by 167
  26. 2023Large-scale Multi-Modal Pre-trained Models: A Comprehensive Surveycited by 153
  27. 2020Re-evaluating Evaluation in Text Summarizationcited by 144
  28. 2023We're Afraid Language Models Aren't Modeling Ambiguitycited by 124
  29. 2023A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasetscited by 123
  30. 2023Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendationcited by 113
  31. 2023Red teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicitycited by 86
  32. 2023Large language models effectively leverage document-level context for literary translation, but critical errors persistcited by 80
  33. 2021BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Modelscited by 76
  34. 2023Is ChatGPT a Good Causal Reasoner? A Comprehensive Evaluationcited by 69
  35. 2023Evaluating Language Models for Mathematics through Interactionscited by 69
  36. 2023The Self-Perception and Political Biases of ChatGPTcited by 66
  37. 2023Boosting Theory-of-Mind Performance in Large Language Models via Promptingcited by 64
  38. 2023Can we trust the evaluation on ChatGPT?cited by 63

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 18; 2024: 27; 2025: 23; 2026: 5.

73 papers.

Sort:

By area:

  1. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 162
  2. 2023Jatmo: Prompt Injection Defense by Task-Specific Finetuningcited by 89
  3. 2023Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?cited by 87
  4. 2024A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendationscited by 84
  5. 2024Infinite-LLM: Efficient LLM Service for Long Context with DistAttention and Distributed KVCachecited by 71
  6. 2023MechAgents: Large language model multi-agent collaborations can solve mechanics problems, generate new data, and integrate knowledgecited by 62
  7. 2023From Concept to Manufacturing: Evaluating Vision-Language Models for Engineering Designcited by 51
  8. 2024A Comprehensive Evaluation of Quantization Strategies for Large Language Modelscited by 47
  9. 2024NuNER: Entity Recognition Encoder Pre-training via LLM-Annotated Datacited by 34
  10. 2023A Wide Evaluation of ChatGPT on Affective Computing Taskscited by 30
  11. 2024PRE: A Peer Review Based Large Language Model Evaluatorcited by 28
  12. 2024Navigating LLM Ethics: Advancements, Challenges, and Future Directionscited by 28
38 more
  1. 2025RbFT: Robust Fine-tuning for Retrieval-Augmented Generation against Retrieval Defectscited by 27
  2. 2024LightVA: Lightweight Visual Analytics with LLM Agent-Based Task Planning and Executioncited by 20
  3. 2024Salute the Classic: Revisiting Challenges of Machine Translation in the Age of Large Language Modelscited by 17
  4. 2023BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Modelscited by 16
  5. 2023From Beginner to Expert: Modeling Medical Knowledge into General LLMscited by 14
  6. 2024Towards Faithful Natural Language Explanations: A Study Using Activation Patching in Large Language Modelscited by 14
  7. 2023CycleAlign: Iterative Distillation from Black-box LLM to White-box Models for Better Human Alignmentcited by 13
  8. 2024See Further for Parameter Efficient Fine-tuning by Standing on the Shoulders of Decompositioncited by 13
  9. 2025Bone Soups: A Seek-and-Soup Model Merging Approach for Controllable Multi-Objective Generationcited by 13
  10. 2024An Examination on the Effectiveness of Divide-and-Conquer Prompting in Large Language Modelscited by 12
  11. 2023An Assessment of ChatGPT on Log Datacited by 11
  12. 2024Benchmarking Large Language Models for Persian: A Preliminary Study Focusing on ChatGPTcited by 11
  13. 2023A Comprehensive Review on Financial Explainable AIcited by 10
  14. 2025A Causality-aware Paradigm for Evaluating Creativity of Multimodal Large Language Modelscited by 10
  15. 2024Explainable Natural Language Processing for Corporate Sustainability Analysiscited by 9
  16. 2023Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Reviewcited by 8
  17. 2023Towards a Psychological Generalist AI: A Survey of Current Applications of Large Language Models and Future Prospectscited by 8
  18. 2024Visual Text Meets Low-level Vision: A Comprehensive Survey on Visual Text Processingcited by 8
  19. 2026Position: LLMs Must Use Functor-Based and RAG-Driven Bias Mitigation for Fairnesscited by 8
  20. 2025JEEM: Vision-Language Understanding in Four Arabic Dialectscited by 7
  21. 2023From Big to Small Without Losing It All: Text Augmentation with ChatGPT for Efficient Sentiment Analysiscited by 6
  22. 2024Generative AI voting: fair collective choice is resilient to LLM biases and inconsistenciescited by 6
  23. 2024CodeGraph: Enhancing Graph Reasoning of LLMs with Codecited by 6
  24. 2025Exploring Cognitive and Aesthetic Causality for Multimodal Aspect-Based Sentiment Analysiscited by 6
  25. 2024Reasoning in Conversation: Solving Subjective Tasks through Dialogue Simulation for Large Language Modelscited by 5
  26. 2024Sparsity-Accelerated Training for Large Language Modelscited by 5
  27. 2025Emotions in the Loop: A Survey of Affective Computing for Emotional Supportcited by 5
  28. 2025Benchmarking GPT-5 for Zero-Shot Multimodal Medical Reasoning in Radiology and Radiation Oncologycited by 5
  29. 2023A Survey on Image-text Multimodal Modelscited by 4
  30. 2024Fine-tuned Large Language Models (LLMs): Improved Prompt Injection Attacks Detectioncited by 4
  31. 2025REGen: A Reliable Evaluation Framework for Generative Event Argument Extractioncited by 4
  32. 2025Mitigating Jailbreaks with Intent-Aware LLMscited by 4
  33. 2023ODSum: New Benchmarks for Open Domain Multi-Document Summarizationcited by 3
  34. 2023Adaptive Training Meets Progressive Scaling: Elevating Efficiency in Diffusion Modelscited by 3
  35. 2024LlamBERT: Large-scale low-cost data annotation in NLPcited by 3
  36. 2024A3D: Does Diffusion Dream about 3D Alignment?cited by 3
  37. 2024AnnotatedTables: A Large Tabular Dataset with Language Model Annotationscited by 3
  38. 2025Perspective Transition of Large Language Models for Solving Subjective Taskscited by 3

50 most cited of 73 All 73 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models
  2. 2024Cross-Language Assessment of Mathematical Capability of ChatGPT
  3. 2023Consistency Analysis of ChatGPT
  4. 2023ChatGPT is a Remarkable Tool -- For Experts
  5. 2023Gpt-4: A Review on Advancements and Opportunities in Natural Language Processing
  6. 2023Complementary Advantages of ChatGPTs and Human Readers in Reasoning: Evidence from English Text Reading Comprehension
  7. 2024Beware of Words: Evaluating the Lexical Diversity of Conversational LLMs using ChatGPT as Case Study
  8. 2023How is ChatGPT's behavior changing over time?