# Assessing Good, Bad and Ugly Arguments Generated by ChatGPT: a New Dataset, its Methodology and Associated Tasks

arXiv: https://arxiv.org/abs/2406.15130 · HTML: https://mlsanity.com/paper/2406.15130/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 23 references that are in the map.
- First posted on arXiv 21 Jun 2024.

## Builds on (most cited first, 8 of 8)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Tasks](https://mlsanity.com/paper/2306.07899/) (2023)
- [Parsing Argumentation Structures in Persuasive Essays](https://mlsanity.com/paper/1604.07370/) (2016)
- [ChatGPT-4 Outperforms Experts and Crowd Workers in Annotating Political Twitter Messages with Zero-Shot Learning](https://mlsanity.com/paper/2304.06588/) (2023)
- [ArgRewrite V.2: an Annotated Argumentative Revisions Corpus](https://mlsanity.com/paper/2206.01677/) (2022)

## Built on by (most cited first, 3 of 3)

- [Large Language Models in Argument Mining: A Survey](https://mlsanity.com/paper/2506.16383/) (2025)
- [An LLM-Based System for Argument Mining](https://mlsanity.com/paper/2605.13793/) (2026)
- [Towards Robust Argumentative Essay Understanding via TIDE: An Interactive Framework with Trial and Debate](https://mlsanity.com/paper/2605.17247/) (2026)

## Abstract (arXiv)

The recent success of Large Language Models (LLMs) has sparked concerns about their potential to spread misinformation. As a result, there is a pressing need for tools to identify ``fake arguments'' generated by such models. To create these tools, examples of texts generated by LLMs are needed. This paper introduces a methodology to obtain good, bad and ugly arguments from argumentative essays produced by ChatGPT, OpenAI's LLM. We then describe a novel dataset containing a set of diverse arguments, ArGPT. We assess the effectiveness of our dataset and establish baselines for several argumentation-related tasks. Finally, we show that the artificially generated data relates well to human argumentation and thus is useful as a tool to train and test systems for the defined tasks.