mlsanityData to 7 Oct 2026

Argument Summarization and its Evaluation in the Era of Large Language Models

arXiv 2503.00847 ↗First posted 2 Mar 2025cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have revolutionized various Natural Language Generation (NLG) tasks, including Argument Summarization (ArgSum), a key subfield of Argument Mining. This paper investigates the integration of state-of-the-art LLMs into ArgSum systems and their evaluation. In particular, we propose a novel prompt-based evaluation scheme, and validate it through a novel human benchmark dataset. Our work makes three main contributions: (i) the integration of LLMs into existing ArgSum systems, (ii) the development of two new LLM-based ArgSum systems, benchmarked against prior methods, and (iii) the introduction of an advanced LLM-based evaluation scheme. We demonstrate that the use of LLMs substantially improves both the generation and evaluation of argument summaries, achieving state-of-the-art results and advancing the field of ArgSum. We also show that among the four LLMs integrated in (i) and (ii), Qwen-3-32B, despite having the fewest parameters, performs best, even surpassing GPT-4o.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 29 of its 73 references that are in the map. First posted on arXiv 2 Mar 2025.

Builds on

29 of 73 references.

References · 29

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  3. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  4. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  5. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  6. 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 779
  7. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  8. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 416
  9. 2019MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distancecited by 415
  10. 2023JudgeLM: Fine-tuned Large Language Models are Scalable Judgescited by 351
  11. 2023PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimizationcited by 311
  12. 2020Evaluation of Text Generation: A Surveycited by 275
17 more
  1. 2017Why We Need New Evaluation Metrics for NLGcited by 232
  2. 2023The Devil is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluationcited by 95
  3. 2024Is Temperature the Creativity Parameter of Large Language Models?cited by 92
  4. 2019A Large-scale Dataset for Argument Quality Ranking: Construction and Analysiscited by 66
  5. 2020From Arguments to Key Points: Towards Automatic Argument Summarizationcited by 60
  6. 2019Classification and Clustering of Arguments with Contextualized Word Embeddingscited by 57
  7. 2016Neural Network-Based Abstract Generation for Opinions and Argumentscited by 46
  8. 2020Aspect-Controlled Neural Argument Generationcited by 43
  9. 2020Quantitative Argument Summarization and Beyond: Cross-Domain Key Point Analysiscited by 31
  10. 2017Measuring the Similarity of Sentential Arguments in Dialogcited by 23
  11. 2021Overview of the 2021 Key Point Analysis Shared Taskcited by 23
  12. 2023The Eval4NLP 2023 Shared Task on Prompting Large Language Models as Explainable Metricscited by 23
  13. 2021Key Point Analysis via Contrastive Learning and Extractive Argument Summarizationcited by 16
  14. 2024PrExMe! Large Scale Prompt Exploration of Open Source LLMs for Machine Translation and Summarization Evaluationcited by 15
  15. 2024Which Side Are You On? A Multi-task Dataset for End-to-End Argument Summarisation and Evaluationcited by 13
  16. 2024Enhancing Argument Summarization: Prioritizing Exhaustiveness in Key Point Generation and Introducing an Automatic Coverage Evaluation Metriccited by 5
  17. 2024PromptOptMe: Error-Aware Prompt Compression for LLM-based MT Evaluation Metricscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025Large Language Models in Argument Mining: A Surveycited by 14
  2. 2025ArgCMV: An Argument Summarization Benchmark for the LLM-eracited by 3
  3. 2025Arg-LLaDA: Argument Summarization via Large Language Diffusion Models and Sufficiency-Aware Refinementcited by 0
  4. 2026Key Point Analysis Needs Structure Recovery: Task Definition, Dataset Diagnosis, and a Structure-Aware Benchmarkcited by 0