# Are Chatbots Reliable Text Annotators? Sometimes

arXiv: https://arxiv.org/abs/2311.05769 · HTML: https://mlsanity.com/paper/2311.05769/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 14 references that are in the map.
- First posted on arXiv 9 Nov 2023.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [Unveiling Security, Privacy, and Ethical Concerns of ChatGPT](https://mlsanity.com/paper/2307.14192/) (2023)
- [Open, Closed, or Small Language Models for Text Classification?](https://mlsanity.com/paper/2308.10092/) (2023)
- [Making Large Language Models Better Data Creators](https://mlsanity.com/paper/2310.20111/) (2023)

## Built on by (most cited first, 9 of 9)

- [Knowledge Distillation in Automated Annotation: Supervised Text Classification with LLM-Generated Training Labels](https://mlsanity.com/paper/2406.17633/) (2024)
- [Harmful YouTube Video Detection: A Taxonomy of Online Harm and MLLMs as Alternative Annotators](https://mlsanity.com/paper/2411.05854/) (2024)
- [Measuring Human Value Expression in Social Media Texts: Calibrated LLM Annotation and Encoder Transfer](https://mlsanity.com/paper/2606.11018/) (2026)
- [Prediction-powered estimators for finite population statistics in highly imbalanced textual data: Public hate crime estimation](https://mlsanity.com/paper/2505.04643/) (2025)
- [Measuring Negative Campaigning across Languages with Large Language Models: A Study of 18 Million Tweets in 19 Countries](https://mlsanity.com/paper/2507.17636/) (2025)
- [How Persuasive is Your Context?](https://mlsanity.com/paper/2509.17879/) (2025)
- [Labeling Training Data for Entity Matching Using Large Language Models](https://mlsanity.com/paper/2606.28823/) (2026)
- [LLM-Assisted Stance Detection in Scientific Discourse: A Test Case in Bayesian Cognitive Science](https://mlsanity.com/paper/2606.15566/) (2026)
- [Human-LLM Collaborative Inductive Coding for Conceptualizing K-12 Educator AI Use](https://mlsanity.com/paper/2607.28889/) (2026)

## Abstract (arXiv)

Recent research highlights the significant potential of ChatGPT for text annotation in social science research. However, ChatGPT is a closed-source product which has major drawbacks with regards to transparency, reproducibility, cost, and data protection. Recent advances in open-source (OS) large language models (LLMs) offer an alternative without these drawbacks. Thus, it is important to evaluate the performance of OS LLMs relative to ChatGPT and standard approaches to supervised machine learning classification. We conduct a systematic comparative evaluation of the performance of a range of OS LLMs alongside ChatGPT, using both zero- and few-shot learning as well as generic and custom prompts, with results compared to supervised classification models. Using a new dataset of tweets from US news media, and focusing on simple binary text annotation tasks, we find significant variation in the performance of ChatGPT and OS models across the tasks, and that the supervised classifier using DistilBERT generally outperforms both. Given the unreliable performance of ChatGPT and the significant challenges it poses to Open Science we advise caution when using ChatGPT for substantive text annotation tasks.