# ArgCMV: An Argument Summarization Benchmark for the LLM-era

arXiv: https://arxiv.org/abs/2508.19580 · HTML: https://mlsanity.com/paper/2508.19580/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 54 references that are in the map.
- First posted on arXiv 27 Aug 2025.

## Builds on (most cited first, 28 of 28)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [Benchmarking Large Language Models for News Summarization](https://mlsanity.com/paper/2301.13848/) (2023)
- [Small Language Models are the Future of Agentic AI](https://mlsanity.com/paper/2506.02153/) (2025)
- [BooookScore: A systematic exploration of book-length summarization in the era of LLMs](https://mlsanity.com/paper/2310.00785/) (2023)
- [Winning Arguments: Interaction Dynamics and Persuasion Strategies in Good-faith Online Discussions](https://mlsanity.com/paper/1602.01103/) (2016)
- [A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methods](https://mlsanity.com/paper/2403.02901/) (2024)
- [A Large-scale Dataset for Argument Quality Ranking: Construction and Analysis](https://mlsanity.com/paper/1911.11408/) (2019)
- [From Arguments to Key Points: Towards Automatic Argument Summarization](https://mlsanity.com/paper/2005.01619/) (2020)
- [DebateSum: A large-scale argument mining and summarization dataset](https://mlsanity.com/paper/2011.07251/) (2020)
- [Quantitative Argument Summarization and Beyond: Cross-Domain Key Point Analysis](https://mlsanity.com/paper/2010.05369/) (2020)
- [SLM-Mod: Small Language Models Surpass LLMs at Content Moderation](https://mlsanity.com/paper/2410.13155/) (2024)
- [Overview of the 2021 Key Point Analysis Shared Task](https://mlsanity.com/paper/2110.10577/) (2021)
- [IAM: A Comprehensive and Large-Scale Dataset for Integrated Argument Mining Tasks](https://mlsanity.com/paper/2203.12257/) (2022)
- [Small Language Models can Outperform Humans in Short Creative Writing: A Study Comparing SLMs with Humans and LLMs](https://mlsanity.com/paper/2409.11547/) (2024)
- [Key Point Analysis via Contrastive Learning and Extractive Argument Summarization](https://mlsanity.com/paper/2109.15086/) (2021)
- [Proactive Moderation of Online Discussions: Existing Practices and the Potential for Algorithmic Support](https://mlsanity.com/paper/2211.16525/) (2022)
- [Which Side Are You On? A Multi-task Dataset for End-to-End Argument Summarisation and Evaluation](https://mlsanity.com/paper/2406.03151/) (2024)
- [Team Enigma at ArgMining-EMNLP 2021: Leveraging Pre-trained Language Models for Key Point Matching](https://mlsanity.com/paper/2110.12370/) (2021)
- [AQE: Argument Quadruplet Extraction via a Quad-Tagging Augmented Generative Approach](https://mlsanity.com/paper/2305.19902/) (2023)
- [OpenDebateEvidence: A Massive-Scale Argument Mining and Summarization Dataset](https://mlsanity.com/paper/2406.14657/) (2024)
- [Do You Hear The People Sing? Key Point Analysis via Iterative Clustering and Abstractive Summarisation](https://mlsanity.com/paper/2305.16000/) (2023)
- [Exploring Key Point Analysis with Pairwise Generation and Graph Partitioning](https://mlsanity.com/paper/2404.11384/) (2024)
- [Argument Summarization and its Evaluation in the Era of Large Language Models](https://mlsanity.com/paper/2503.00847/) (2025)
- [A Novel LLM-based Two-stage Summarization Approach for Long Dialogues](https://mlsanity.com/paper/2410.06520/) (2024)
- [Needling Through the Threads: A Visualization Tool for Navigating Threaded Online Discussions](https://mlsanity.com/paper/2506.11276/) (2025)

## Built on by (most cited first, 3 of 3)

- [PERSPECTRA: A Scalable and Configurable Pluralist Benchmark of Perspectives from Arguments](https://mlsanity.com/paper/2602.08716/) (2026)
- [ArgBench: Benchmarking LLMs on Computational Argumentation Tasks](https://mlsanity.com/paper/2604.17366/) (2026)
- [Key Point Analysis Needs Structure Recovery: Task Definition, Dataset Diagnosis, and a Structure-Aware Benchmark](https://mlsanity.com/paper/2608.25854/) (2026)

## Abstract (arXiv)

Key point extraction is an important task in argument summarization which involves extracting high-level short summaries from arguments. Existing approaches for KP extraction have been mostly evaluated on the popular ArgKP21 dataset. In this paper, we highlight some of the major limitations of the ArgKP21 dataset and demonstrate the need for new benchmarks that are more representative of actual human conversations. Using SoTA large language models (LLMs), we curate a new argument key point extraction dataset called ArgCMV comprising of around 12K arguments from actual online human debates spread across over 3K topics. Our dataset exhibits higher complexity such as longer, co-referencing arguments, higher presence of subjective discourse units, and a larger range of topics over ArgKP21. We show that existing methods do not adapt well to ArgCMV and provide extensive benchmark results by experimenting with existing baselines and latest open source models. This work introduces a novel KP extraction dataset for long-context online discussions, setting the stage for the next generation of LLM-driven summarization research.