# Automatic Argument Quality Assessment -- New Datasets and Methods

arXiv: https://arxiv.org/abs/1909.01007 · HTML: https://mlsanity.com/paper/1909.01007/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Sep 2019.

## Built on by (most cited first, 30 of 47)

- [What Evidence Do Language Models Find Convincing?](https://mlsanity.com/paper/2402.11782/) (2024)
- [TWEETSUMM -- A Dialog Summarization Dataset for Customer Service](https://mlsanity.com/paper/2111.11894/) (2021)
- [Can LLMs Speak For Diverse People? Tuning LLMs via Debate to Generate Controllable Controversial Statements](https://mlsanity.com/paper/2402.10614/) (2024)
- [How Susceptible are LLMs to Influence in Prompts?](https://mlsanity.com/paper/2408.11865/) (2024)
- [Measuring and Benchmarking Large Language Models' Capabilities to Generate Persuasive Language](https://mlsanity.com/paper/2406.17753/) (2024)
- [Are Large Language Models Reliable Argument Quality Annotators?](https://mlsanity.com/paper/2404.09696/) (2024)
- [Overview of the 2021 Key Point Analysis Shared Task](https://mlsanity.com/paper/2110.10577/) (2021)
- [IAM: A Comprehensive and Large-Scale Dataset for Integrated Argument Mining Tasks](https://mlsanity.com/paper/2203.12257/) (2022)
- [Every Bite Is an Experience: Key Point Analysis of Business Reviews](https://mlsanity.com/paper/2106.06758/) (2021)
- [Fair and Argumentative Language Modeling for Computational Argumentation](https://mlsanity.com/paper/2204.04026/) (2022)
- [Exploring Jiu-Jitsu Argumentation for Writing Peer Review Rebuttals](https://mlsanity.com/paper/2311.03998/) (2023)
- [Scientia Potentia Est -- On the Role of Knowledge in Computational Argumentation](https://mlsanity.com/paper/2107.00281/) (2021)
- [Key Point Analysis via Contrastive Learning and Extractive Argument Summarization](https://mlsanity.com/paper/2109.15086/) (2021)
- [Learning From Revisions: Quality Assessment of Claims in Argumentation at Scale](https://mlsanity.com/paper/2101.10250/) (2021)
- [Large Language Models in Argument Mining: A Survey](https://mlsanity.com/paper/2506.16383/) (2025)
- [MultiOpEd: A Corpus of Multi-Perspective News Editorials](https://mlsanity.com/paper/2106.02725/) (2021)
- [Which Side Are You On? A Multi-task Dataset for End-to-End Argument Summarisation and Evaluation](https://mlsanity.com/paper/2406.03151/) (2024)
- [Must Read: A Comprehensive Survey of Computational Persuasion](https://mlsanity.com/paper/2505.07775/) (2025)
- [Assessing the Sufficiency of Arguments through Conclusion Generation](https://mlsanity.com/paper/2110.13495/) (2021)
- [LLM-based Rewriting of Inappropriate Argumentation using Reinforcement Learning from Machine Feedback](https://mlsanity.com/paper/2406.03363/) (2024)
- [Persuasiveness of Generated Free-Text Rationales in Subjective Decisions: A Case Study on Pairwise Argument Ranking](https://mlsanity.com/paper/2406.13905/) (2024)
- [Do Emotions Really Affect Argument Convincingness? A Dynamic Approach with LLM-based Manipulation Checks](https://mlsanity.com/paper/2503.00024/) (2025)
- [Summary Grounded Conversation Generation](https://mlsanity.com/paper/2106.03337/) (2021)
- [An Item Response Theory Framework for Persuasion](https://mlsanity.com/paper/2204.11337/) (2022)
- [Do You Hear The People Sing? Key Point Analysis via Iterative Clustering and Abstractive Summarisation](https://mlsanity.com/paper/2305.16000/) (2023)
- [Not all Fake News is Written: A Dataset and Analysis of Misleading Video Headlines](https://mlsanity.com/paper/2310.13859/) (2023)
- [Communication is All You Need: Persuasion Dataset Construction via Multi-LLM Communication](https://mlsanity.com/paper/2502.08896/) (2025)
- [Towards a Perspectivist Turn in Argument Quality Assessment](https://mlsanity.com/paper/2502.14501/) (2025)
- [Contextualizing Argument Quality Assessment with Relevant Knowledge](https://mlsanity.com/paper/2305.12280/) (2023)
- [How to Handle Different Types of Out-of-Distribution Scenarios in Computational Argumentation? A Comprehensive and Fine-Grained Field Study](https://mlsanity.com/paper/2309.08316/) (2023)

## Abstract (arXiv)

We explore the task of automatic assessment of argument quality. To that end, we actively collected 6.3k arguments, more than a factor of five compared to previously examined data. Each argument was explicitly and carefully annotated for its quality. In addition, 14k pairs of arguments were annotated independently, identifying the higher quality argument in each pair. In spite of the inherent subjective nature of the task, both annotation schemes led to surprisingly consistent results. We release the labeled datasets to the community. Furthermore, we suggest neural methods based on a recently released language model, for argument ranking as well as for argument-pair classification. In the former task, our results are comparable to state-of-the-art; in the latter task our results significantly outperform earlier methods.