Assessing Good, Bad and Ugly Arguments Generated by ChatGPT: a New Dataset, its Methodology and Associated Tasks
Not on the map. Why?
Abstract (arXiv)
The recent success of Large Language Models (LLMs) has sparked concerns about their potential to spread misinformation. As a result, there is a pressing need for tools to identify ``fake arguments'' generated by such models. To create these tools, examples of texts generated by LLMs are needed. This paper introduces a methodology to obtain good, bad and ugly arguments from argumentative essays produced by ChatGPT, OpenAI's LLM. We then describe a novel dataset containing a set of diverse arguments, ArGPT. We assess the effectiveness of our dataset and establish baselines for several argumentation-related tasks. Finally, we show that the artificially generated data relates well to human argumentation and thus is useful as a tool to train and test systems for the defined tasks.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 8 of its 23 references that are in the map. First posted on arXiv 21 Jun 2024.
Builds on
8 of 23 references.
References · 8
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
- 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 849
- 2023Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Taskscited by 102
- 2016Parsing Argumentation Structures in Persuasive Essayscited by 99
- 2023ChatGPT-4 Outperforms Experts and Crowd Workers in Annotating Political Twitter Messages with Zero-Shot Learningcited by 97
- 2022ArgRewrite V.2: an Annotated Argumentative Revisions Corpuscited by 9
Built on by
Citing papers per year: 2025: 1; 2026: 2.
3 papers.