# Which Side Are You On? A Multi-task Dataset for End-to-End Argument Summarisation and Evaluation

arXiv: https://arxiv.org/abs/2406.03151 · HTML: https://mlsanity.com/paper/2406.03151/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 91 references that are in the map.
- First posted on arXiv 5 Jun 2024.

## Builds on (most cited first, 30 of 37)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://mlsanity.com/paper/2108.08877/) (2021)
- [Parsing Argumentation Structures in Persuasive Essays](https://mlsanity.com/paper/1604.07370/) (2016)
- [A Large-scale Dataset for Argument Quality Ranking: Construction and Analysis](https://mlsanity.com/paper/1911.11408/) (2019)
- [From Arguments to Key Points: Towards Automatic Argument Summarization](https://mlsanity.com/paper/2005.01619/) (2020)
- [Classification and Clustering of Arguments with Contextualized Word Embeddings](https://mlsanity.com/paper/1906.09821/) (2019)
- [Automatic Argument Quality Assessment -- New Datasets and Methods](https://mlsanity.com/paper/1909.01007/) (2019)
- [ArguGPT: evaluating, understanding and identifying argumentative essays generated by GPT models](https://mlsanity.com/paper/2304.07666/) (2023)
- [Corpus Wide Argument Mining -- a Working Solution](https://mlsanity.com/paper/1911.10763/) (2019)
- [DebateSum: A large-scale argument mining and summarization dataset](https://mlsanity.com/paper/2011.07251/) (2020)
- [What is the Essence of a Claim? Cross-Domain Claim Identification](https://mlsanity.com/paper/1704.07203/) (2017)
- [Quantitative Argument Summarization and Beyond: Cross-Domain Key Point Analysis](https://mlsanity.com/paper/2010.05369/) (2020)
- [Are You Convinced? Choosing the More Convincing Evidence with a Siamese Network](https://mlsanity.com/paper/1907.08971/) (2019)
- [Fine-Grained Argument Unit Recognition and Classification](https://mlsanity.com/paper/1904.09688/) (2019)
- [Exploring the Role of Argument Structure in Online Debate Persuasion](https://mlsanity.com/paper/2010.03538/) (2020)
- [Measuring the Similarity of Sentential Arguments in Dialog](https://mlsanity.com/paper/1709.01887/) (2017)
- [Overview of the 2021 Key Point Analysis Shared Task](https://mlsanity.com/paper/2110.10577/) (2021)
- [IAM: A Comprehensive and Large-Scale Dataset for Integrated Argument Mining Tasks](https://mlsanity.com/paper/2203.12257/) (2022)
- [Multilingual Argument Mining: Datasets and Analysis](https://mlsanity.com/paper/2010.06432/) (2020)
- [Key Point Analysis via Contrastive Learning and Extractive Argument Summarization](https://mlsanity.com/paper/2109.15086/) (2021)

## Built on by (most cited first, 13 of 13)

- [Large Language Models in Argument Mining: A Survey](https://mlsanity.com/paper/2506.16383/) (2025)
- [Towards a Perspectivist Turn in Argument Quality Assessment](https://mlsanity.com/paper/2502.14501/) (2025)
- [Argument Summarization and its Evaluation in the Era of Large Language Models](https://mlsanity.com/paper/2503.00847/) (2025)
- [VTechAGP: An Academic-to-General-Audience Text Paraphrase Dataset and Benchmark Models](https://mlsanity.com/paper/2411.04825/) (2024)
- [Sci-LoRA: Mixture of Scientific LoRAs for Cross-Domain Lay Paraphrasing](https://mlsanity.com/paper/2505.18867/) (2025)
- [AMELIA: A Family of Multi-task End-to-end Language Models for Argumentation](https://mlsanity.com/paper/2508.17926/) (2025)
- [ArgCMV: An Argument Summarization Benchmark for the LLM-era](https://mlsanity.com/paper/2508.19580/) (2025)
- [Learning to Generate and Evaluate Fact-checking Explanations with Transformers](https://mlsanity.com/paper/2410.15669/) (2024)
- [PerSphere: A Comprehensive Framework for Multi-Faceted Perspective Retrieval and Summarization](https://mlsanity.com/paper/2412.12588/) (2024)
- [Evaluating LLM-Driven Summarisation of Parliamentary Debates with Computational Argumentation](https://mlsanity.com/paper/2604.19331/) (2026)
- [Arg-LLaDA: Argument Summarization via Large Language Diffusion Models and Sufficiency-Aware Refinement](https://mlsanity.com/paper/2507.19081/) (2025)
- [Can One-sided Arguments Lead to Response Change in Large Language Models?](https://mlsanity.com/paper/2602.06260/) (2026)
- [An LLM-Based System for Argument Mining](https://mlsanity.com/paper/2605.13793/) (2026)

## Abstract (arXiv)

With the recent advances of large language models (LLMs), it is no longer infeasible to build an automated debate system that helps people to synthesise persuasive arguments. Previous work attempted this task by integrating multiple components. In our work, we introduce an argument mining dataset that captures the end-to-end process of preparing an argumentative essay for a debate, which covers the tasks of claim and evidence identification (Task 1 ED), evidence convincingness ranking (Task 2 ECR), argumentative essay summarisation and human preference ranking (Task 3 ASR) and metric learning for automated evaluation of resulting essays, based on human feedback along argument quality dimensions (Task 4 SQE). Our dataset contains 14k examples of claims that are fully annotated with the various properties supporting the aforementioned tasks. We evaluate multiple generative baselines for each of these tasks, including representative LLMs. We find, that while they show promising results on individual tasks in our benchmark, their end-to-end performance on all four tasks in succession deteriorates significantly, both in automated measures as well as in human-centred evaluation. This challenge presented by our proposed dataset motivates future research on end-to-end argument mining and summarisation. The repository of this project is available at https://github.com/HaoBytes/ArgSum-Datatset