mlsanityData to 14 Oct 2026

A Large-scale Dataset for Argument Quality Ranking: Construction and Analysis

arXiv 1911.11408 ↗First posted 26 Nov 2019cs.CLcited by 67

Not on the map. Why?

Abstract (arXiv)

Identifying the quality of free-text arguments has become an important task in the rapidly expanding field of computational argumentation. In this work, we explore the challenging task of argument quality ranking. To this end, we created a corpus of 30,497 arguments carefully annotated for point-wise quality, released as part of this work. To the best of our knowledge, this is the largest dataset annotated for point-wise argument quality, larger by a factor of five than previously released datasets. Moreover, we address the core issue of inducing a labeled score from crowd annotations by performing a comprehensive evaluation of different approaches to this problem. In addition, we analyze the quality dimensions that characterize this dataset. Finally, we present a neural method for argument quality ranking, which outperforms several baselines on our own dataset, as well as previous methods published for another dataset.

Key facts. Cited by 67 papers in the map (data through 14 Oct 2026). First posted on arXiv 26 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 12; 2022: 8; 2023: 8; 2024: 11; 2025: 14; 2026: 14.

67 papers.

Sort:

By area:

  1. 2024What Evidence Do Language Models Find Convincing?cited by 75
  2. 2021ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoningcited by 58
  3. 2024Can LLMs Speak For Diverse People? Tuning LLMs via Debate to Generate Controllable Controversial Statementscited by 34
  4. 2023The Touch'e23-ValueEval Dataset for Identifying Human Values behind Argumentscited by 28
  5. 2021Overview of the 2021 Key Point Analysis Shared Taskcited by 23
  6. 2022IAM: A Comprehensive and Large-Scale Dataset for Integrated Argument Mining Taskscited by 23
  7. 2021Every Bite Is an Experience: Key Point Analysis of Business Reviewscited by 19
  8. 2022Fair and Argumentative Language Modeling for Computational Argumentationcited by 18
  9. 2023Exploring Jiu-Jitsu Argumentation for Writing Peer Review Rebuttalscited by 18
  10. 2021Scientia Potentia Est -- On the Role of Knowledge in Computational Argumentationcited by 17
  11. 2024Lies, Damned Lies, and Distributional Language Statistics: Persuasion and Deception with Large Language Modelscited by 15
  12. 2021Learning From Revisions: Quality Assessment of Claims in Argumentation at Scalecited by 14
38 more
  1. 2025Large Language Models in Argument Mining: A Surveycited by 14
  2. 2024Which Side Are You On? A Multi-task Dataset for End-to-End Argument Summarisation and Evaluationcited by 13
  3. 2025Must Read: A Comprehensive Survey of Computational Persuasioncited by 13
  4. 2021Team Enigma at ArgMining-EMNLP 2021: Leveraging Pre-trained Language Models for Key Point Matchingcited by 11
  5. 2021Assessing the Sufficiency of Arguments through Conclusion Generationcited by 11
  6. 2021Project Debater APIs: Decomposing the AI Grand Challengecited by 8
  7. 2023To Revise or Not to Revise: Learning to Detect Improvable Claims for Argumentative Writing Supportcited by 8
  8. 2024Political DEBATE: Efficient Zero-shot and Few-shot Classifiers for Political Textcited by 8
  9. 2021DeepA2: A Modular Framework for Deep Argument Analysis with Pretrained Neural Text2Text Language Modelscited by 7
  10. 2022Will It Blend? Mixing Training Paradigms & Prompting for Argument Quality Predictioncited by 7
  11. 2024Persuasiveness of Generated Free-Text Rationales in Subjective Decisions: A Case Study on Pairwise Argument Rankingcited by 7
  12. 2023Adam-Smith at SemEval-2023 Task 4: Discovering Human Values in Arguments with Ensembles of Transformer-based Modelscited by 6
  13. 2024A Hybrid Intelligence Method for Argument Miningcited by 6
  14. 2024Prompted Aspect Key Point Analysis for Quantitative Review Summarizationcited by 6
  15. 2024From Facts to Insights: A Study on the Generation and Evaluation of Analytical Reports for Deciphering Earnings Callscited by 6
  16. 2022An Item Response Theory Framework for Persuasioncited by 5
  17. 2023Architectural Sweet Spots for Modeling Human Label Variation by the Example of Argument Quality: It's Best to Relate Perspectives!cited by 5
  18. 2025Towards a Perspectivist Turn in Argument Quality Assessmentcited by 5
  19. 2023Contextualizing Argument Quality Assessment with Relevant Knowledgecited by 4
  20. 2024Argument Quality Assessment in the Age of Instruction-Following Large Language Modelscited by 4
  21. 2025Argument Summarization and its Evaluation in the Era of Large Language Modelscited by 4
  22. 2025ArgInstruct: Specialized Instruction Fine-Tuning for Computational Argumentationcited by 4
  23. 2025Debatable Intelligence: Benchmarking LLM Judges via Debate Speech Evaluationcited by 3
  24. 2025ArgCMV: An Argument Summarization Benchmark for the LLM-eracited by 3
  25. 2022Towards a Holistic View on Argument Quality Predictioncited by 2
  26. 2024When LLMs are Unfit Use FastFit: Fast and Effective Text Classification with Many Classescited by 2
  27. 2025Investigating Subjective Factors of Argument Strength: Storytelling, Emotions, and Hedgingcited by 2
  28. 2025Trust Me, I Can Convince You: The Contextualized Argument Appraisal Frameworkcited by 2
  29. 2026Belief Engine: Configurable and Inspectable Stance Dynamics in Multi-Agent LLM Deliberationcited by 2
  30. 2026More Choices, Fewer Decisions: Ordinal-Scale Bias in JEV-like Direct-Decision Modelscited by 2
  31. 2022Arguments to Key Points Mapping with Prompt-based Learningcited by 1
  32. 2023Conclusion-based Counter-Argument Generationcited by 1
  33. 2023Multi-Task Learning Improves Performance In Deep Argument Mining Modelscited by 1
  34. 2025A Generalizable Rhetorical Strategy Annotation Model Using LLM-based Debate Simulation and Labellingcited by 1
  35. 2026Contradiction to Consensus: Dual Perspective, Multi Source Retrieval Based Claim Verification with Source Level Disagreement using LLMcited by 1
  36. 2026DEFINED: A Data-Efficient Computational Framework for Fine-Grained Creativity Assessment in Debate Scenarioscited by 1
  37. 2026A dataset of rated conceptual argumentscited by 1
  38. 2021A Comparative Study on Collecting High-Quality Implicit Reasonings at a Large-scalecited by 0

50 most cited of 67 All 67 (JSON)