mlsanityData to 14 Oct 2026

Testing the Reliability of ChatGPT for Text Annotation and Classification: A Cautionary Remark

arXiv 2304.11085 ↗First posted 17 Apr 2023cs.CLcited by 54

Not on the map. Why?

Abstract (arXiv)

Recent studies have demonstrated promising potential of ChatGPT for various text annotation and classification tasks. However, ChatGPT is non-deterministic which means that, as with human coders, identical input can lead to different outputs. Given this, it seems appropriate to test the reliability of ChatGPT. Therefore, this study investigates the consistency of ChatGPT's zero-shot capabilities for text annotation and classification, focusing on different model parameters, prompt variations, and repetitions of identical inputs. Based on the real-world classification task of differentiating website texts into news and not news, results show that consistency in ChatGPT's classification output can fall short of scientific thresholds for reliability. For example, even minor wording alterations in prompts or repeating the identical input can lead to varying outputs. Although pooling outputs from multiple repetitions can improve reliability, this study advises caution when using ChatGPT for zero-shot text annotation and underscores the need for thorough validation, such as comparison against human-annotated data. The unsupervised application of ChatGPT for text annotation and classification is not recommended.

Key facts. Cited by 54 papers in the map (data through 14 Oct 2026). Builds on 6 of its 16 references that are in the map. First posted on arXiv 17 Apr 2023.

Builds on

6 of 16 references.

References · 6

  1. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 870
  2. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
  3. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 488
  4. 2023Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speechcited by 166
  5. 2023Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERTcited by 155
  6. 2023ChatGPT: Beginning of an End of Manual Linguistic Data Annotation? Use Case of Automatic Genre Identificationcited by 45

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 17; 2024: 20; 2025: 10; 2026: 7.

54 papers.

Sort:

By area:

  1. 2025LLM Social Simulations Are a Promising Research Methodcited by 99
  2. 2023CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotationcited by 94
  3. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 93
  4. 2023Automated Annotation with Generative AI Requires Validationcited by 76
  5. 2023"HOT" ChatGPT: The promise of ChatGPT in detecting and discriminating hateful, offensive, and toxic comments on social mediacited by 45
  6. 2023GPTAraEval: A Comprehensive Evaluation of ChatGPT on Arabic NLPcited by 37
  7. 2023Can GPT-4 Support Analysis of Textual Data in Tasks Requiring Highly Specialized Domain Expertise?cited by 31
  8. 2024FACT-GPT: Fact-Checking Augmentation via Claim Matching with LLMscited by 27
  9. 2024Knowledge Distillation in Automated Annotation: Supervised Text Classification with LLM-Generated Training Labelscited by 25
  10. 2024Under the Surface: Tracking the Artifactuality of LLM-Generated Datacited by 24
  11. 2023Navigating Prompt Complexity for Zero-Shot Classification: A Study of Large Language Models in Computational Social Sciencecited by 19
  12. 2023Automated Claim Matching with Large Language Models: Empowering Fact-Checkers in the Fight Against Misinformationcited by 19
38 more
  1. 2024Large Language Models as Financial Data Annotators: A Study on Effectiveness and Efficiencycited by 18
  2. 2024Keeping Humans in the Loop: Human-Centered Automated Annotation with Generative AIcited by 17
  3. 2023Assessing Student Errors in Experimentation Using Artificial Intelligence and Large Language Models: A Comparative Study with Human Raterscited by 16
  4. 2024Prompt Stability Scoring for Text Annotation with Large Language Modelscited by 15
  5. 2023Automated stance detection in complex topics and small languages: the challenging case of immigration in polarizing news mediacited by 14
  6. 2023Leveraging Large Language Models for Automated Dialogue Analysiscited by 14
  7. 2023A Large Language Model Approach to Educational Survey Feedback Analysiscited by 14
  8. 2023Exploring Recommendation Capabilities of GPT-4V(ision): A Preliminary Case Studycited by 12
  9. 2023Evaluation is all you need. Prompting Generative Large Language Models for Annotation Tasks in the Social Sciences. A Primer using Open Modelscited by 12
  10. 2024ChatGPT Doesn't Trust Chargers Fans: Guardrail Sensitivity in Contextcited by 9
  11. 2023Leveraging Large Language Models and Weak Supervision for Social Media data annotation: an evaluation using COVID-19 self-reported vaccination tweetscited by 8
  12. 2024APT-Pipe: A Prompt-Tuning Tool for Social Data Annotation using ChatGPTcited by 5
  13. 2024Dynamic Sentiment Analysis with Local Large Language Models using Majority Voting: A Study on Factors Affecting Restaurant Evaluationcited by 5
  14. 2024Enhancing Text Annotation through Rationale-Driven Collaborative Few-Shot Promptingcited by 5
  15. 2025AI-induced sexual harassment: Investigating Contextual Characteristics and User Reactions of Sexual Harassment by a Companion Chatbotcited by 5
  16. 2024Evaluating the Performance of ChatGPT for Spam Email Detectioncited by 4
  17. 2024Harnessing Artificial Intelligence to Combat Online Hate: Exploring the Challenges and Opportunities of Large Language Models in Hate Speech Detectioncited by 4
  18. 2024GPT Assisted Annotation of Rhetorical and Linguistic Features for Interpretable Propaganda Technique Detection in News Textcited by 4
  19. 2025Deconstructing Depression Stigma: Integrating AI-driven Data Collection and Analysis with Causal Knowledge Graphscited by 4
  20. 2025Memory Is All You Need: Testing How Model Memory Affects LLM Performance in Annotation Taskscited by 4
  21. 2026Benchmark Illusion: Disagreement among LLMs and Its Scientific Consequencescited by 4
  22. 2023OverPrompt: Enhancing ChatGPT through Efficient In-Context Learningcited by 3
  23. 2023Human-AI Collaboration in Thematic Analysis using ChatGPT: A User Study and Design Recommendationscited by 3
  24. 2026Automatic Prompt Optimization for Dataset-Level Feature Discoverycited by 3
  25. 2024Stability Analysis of ChatGPT-based Sentiment Analysis in AI Quality Assurancecited by 2
  26. 2024Exploring the Human-LLM Synergy in Advancing Theory-driven Qualitative Analysiscited by 2
  27. 2025Introducing HALC: A general pipeline for the systematic and reliable construction of prompts for automated coding with LLMs in the computational social sciencescited by 2
  28. 2026Language Models Agree With Each Other, Not With Readerscited by 2
  29. 2024Framing Social Movements on Social Media: Unpacking Diagnostic, Prognostic, and Motivational Strategiescited by 1
  30. 2024Scaling Technology Acceptance Analysis with Large Language Model (LLM) Annotation Systemscited by 1
  31. 2024Utilizing Large Language Models for Named Entity Recognition in Traditional Chinese Medicine against COVID-19 Literature: Comparative Studycited by 1
  32. 2025Arti-"fickle" Intelligence: Using LLMs as a Tool for Inference in the Political and Social Sciencescited by 1
  33. 2025Reliable Decision Support with LLMs: A Framework for Evaluating Consistency in Binary Text Classification Applicationscited by 1
  34. 2024A Study on the Vulnerability of Test Questions against ChatGPT-based Cheatingcited by 0
  35. 2024From Measurement Instruments to Data: Leveraging Theory-Driven Synthetic Training Data for Classifying Social Constructscited by 0
  36. 2025Tell Me What You Know About Sexism: Expert-LLM Interaction Strategies and Co-Created Definitions for Zero-Shot Sexism Detectioncited by 0
  37. 2025Generative Large Language Models (gLLMs) in Content Analysis: A Practical Guide for Communication Researchcited by 0
  38. 2025A Comparison of Human and ChatGPT Classification Performance on Complex Social Media Datacited by 0

50 most cited of 54 All 54 (JSON)