mlsanityData to 7 Oct 2026

Automatic Argument Quality Assessment -- New Datasets and Methods

arXiv 1909.01007 ↗First posted 3 Sep 2019cs.CLcited by 47

Not on the map. Why?

Abstract (arXiv)

We explore the task of automatic assessment of argument quality. To that end, we actively collected 6.3k arguments, more than a factor of five compared to previously examined data. Each argument was explicitly and carefully annotated for its quality. In addition, 14k pairs of arguments were annotated independently, identifying the higher quality argument in each pair. In spite of the inherent subjective nature of the task, both annotation schemes led to surprisingly consistent results. We release the labeled datasets to the community. Furthermore, we suggest neural methods based on a recently released language model, for argument ranking as well as for argument-pair classification. In the former task, our results are comparable to state-of-the-art; in the latter task our results significantly outperform earlier methods.

Key facts. Cited by 47 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 5; 2023: 6; 2024: 10; 2025: 10; 2026: 6.

47 papers.

Sort:

By area:

  1. 2024What Evidence Do Language Models Find Convincing?cited by 74
  2. 2021TWEETSUMM -- A Dialog Summarization Dataset for Customer Servicecited by 34
  3. 2024Can LLMs Speak For Diverse People? Tuning LLMs via Debate to Generate Controllable Controversial Statementscited by 33
  4. 2024How Susceptible are LLMs to Influence in Prompts?cited by 32
  5. 2024Measuring and Benchmarking Large Language Models' Capabilities to Generate Persuasive Languagecited by 31
  6. 2024Are Large Language Models Reliable Argument Quality Annotators?cited by 24
  7. 2021Overview of the 2021 Key Point Analysis Shared Taskcited by 23
  8. 2022IAM: A Comprehensive and Large-Scale Dataset for Integrated Argument Mining Taskscited by 23
  9. 2021Every Bite Is an Experience: Key Point Analysis of Business Reviewscited by 19
  10. 2022Fair and Argumentative Language Modeling for Computational Argumentationcited by 18
  11. 2023Exploring Jiu-Jitsu Argumentation for Writing Peer Review Rebuttalscited by 18
  12. 2021Scientia Potentia Est -- On the Role of Knowledge in Computational Argumentationcited by 17
35 more
  1. 2021Key Point Analysis via Contrastive Learning and Extractive Argument Summarizationcited by 16
  2. 2021Learning From Revisions: Quality Assessment of Claims in Argumentation at Scalecited by 14
  3. 2025Large Language Models in Argument Mining: A Surveycited by 14
  4. 2021MultiOpEd: A Corpus of Multi-Perspective News Editorialscited by 13
  5. 2024Which Side Are You On? A Multi-task Dataset for End-to-End Argument Summarisation and Evaluationcited by 13
  6. 2025Must Read: A Comprehensive Survey of Computational Persuasioncited by 12
  7. 2021Assessing the Sufficiency of Arguments through Conclusion Generationcited by 11
  8. 2024LLM-based Rewriting of Inappropriate Argumentation using Reinforcement Learning from Machine Feedbackcited by 7
  9. 2024Persuasiveness of Generated Free-Text Rationales in Subjective Decisions: A Case Study on Pairwise Argument Rankingcited by 7
  10. 2025Do Emotions Really Affect Argument Convincingness? A Dynamic Approach with LLM-based Manipulation Checkscited by 6
  11. 2021Summary Grounded Conversation Generationcited by 5
  12. 2022An Item Response Theory Framework for Persuasioncited by 5
  13. 2023Do You Hear The People Sing? Key Point Analysis via Iterative Clustering and Abstractive Summarisationcited by 5
  14. 2023Not all Fake News is Written: A Dataset and Analysis of Misleading Video Headlinescited by 5
  15. 2025Communication is All You Need: Persuasion Dataset Construction via Multi-LLM Communicationcited by 5
  16. 2025Towards a Perspectivist Turn in Argument Quality Assessmentcited by 5
  17. 2023Contextualizing Argument Quality Assessment with Relevant Knowledgecited by 4
  18. 2023How to Handle Different Types of Out-of-Distribution Scenarios in Computational Argumentation? A Comprehensive and Fine-Grained Field Studycited by 4
  19. 2024Argument Quality Assessment in the Age of Instruction-Following Large Language Modelscited by 4
  20. 2025ArgInstruct: Specialized Instruction Fine-Tuning for Computational Argumentationcited by 4
  21. 2025Debatable Intelligence: Benchmarking LLM Judges via Debate Speech Evaluationcited by 3
  22. 2021Active Learning for Argument Strength Estimationcited by 2
  23. 2022Towards a Holistic View on Argument Quality Predictioncited by 2
  24. 2024WIBA: What Is Being Argued? A Comprehensive Approach to Argument Miningcited by 2
  25. 2025Investigating Subjective Factors of Argument Strength: Storytelling, Emotions, and Hedgingcited by 2
  26. 2023Automated Essay Scoring in Argumentative Writing: DeBERTeachingAssistantcited by 1
  27. 2025A Generalizable Rhetorical Strategy Annotation Model Using LLM-based Debate Simulation and Labellingcited by 1
  28. 2026Contradiction to Consensus: Dual Perspective, Multi Source Retrieval Based Claim Verification with Source Level Disagreement using LLMcited by 1
  29. 2026A dataset of rated conceptual argumentscited by 1
  30. 2022VIRATrustData: A Trust-Annotated Corpus of Human-Chatbot Conversations About COVID-19 Vaccinescited by 0
  31. 2025Automated Essay Scoring Incorporating Annotations from Automated Feedback Systemscited by 0
  32. 2026GRASP: Deterministic argument ranking in interaction graphscited by 0
  33. 2026Argument Quality Assessment with Large Language Models: A Pairwise Bradley-Terry Approachcited by 0
  34. 2026Persuasion Index: A Theory-Guided Framework for Persuasion Analysiscited by 0
  35. 2026Logical Embeddings for Argument Analysiscited by 0