# Classification and Clustering of Arguments with Contextualized Word Embeddings

arXiv: https://arxiv.org/abs/1906.09821 · HTML: https://mlsanity.com/paper/1906.09821/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jun 2019.

## Built on by (most cited first, 30 of 57)

- [SGPT: GPT Sentence Embeddings for Semantic Search](https://mlsanity.com/paper/2202.08904/) (2022)
- [SBERT studies Meaning Representations: Decomposing Sentence Embeddings into Explainable Semantic Features](https://mlsanity.com/paper/2206.07023/) (2022)
- [DirectProbe: Studying Representations without Classifiers](https://mlsanity.com/paper/2104.05904/) (2021)
- [ArgLegalSumm: Improving Abstractive Summarization of Legal Documents with Argument Mining](https://mlsanity.com/paper/2209.01650/) (2022)
- [IAM: A Comprehensive and Large-Scale Dataset for Integrated Argument Mining Tasks](https://mlsanity.com/paper/2203.12257/) (2022)
- [Natural Language Decompositions of Implicit Content Enable Better Text Representations](https://mlsanity.com/paper/2305.14583/) (2023)
- [Classifying Argumentative Relations Using Logical Mechanisms and Argumentation Schemes](https://mlsanity.com/paper/2105.07571/) (2021)
- [Scientia Potentia Est -- On the Role of Knowledge in Computational Argumentation](https://mlsanity.com/paper/2107.00281/) (2021)
- [Examining Bias in Opinion Summarisation Through the Perspective of Opinion Diversity](https://mlsanity.com/paper/2306.04424/) (2023)
- [Large Language Models in Argument Mining: A Survey](https://mlsanity.com/paper/2506.16383/) (2025)
- [Which Side Are You On? A Multi-task Dataset for End-to-End Argument Summarisation and Evaluation](https://mlsanity.com/paper/2406.03151/) (2024)
- [Is Stance Detection Topic-Independent and Cross-topic Generalizable? -- A Reproduction Study](https://mlsanity.com/paper/2110.07693/) (2021)
- [SciClops: Detecting and Contextualizing Scientific Claims for Assisting Manual Fact-Checking](https://mlsanity.com/paper/2110.13090/) (2021)
- [Evaluating Pre-Trained Models for User Feedback Analysis in Software Engineering: A Study on Classification of App-Reviews](https://mlsanity.com/paper/2104.05861/) (2021)
- [Deep Transfer Learning & Beyond: Transformer Language Models in Information Systems Research](https://mlsanity.com/paper/2110.08975/) (2021)
- [Persua: A Visual Interactive System to Enhance the Persuasiveness of Arguments in Online Discussion](https://mlsanity.com/paper/2204.07741/) (2022)
- [Will It Blend? Mixing Training Paradigms & Prompting for Argument Quality Prediction](https://mlsanity.com/paper/2209.08966/) (2022)
- [Equiangular Basis Vectors](https://mlsanity.com/paper/2303.11637/) (2023)
- [Multi-Task Attentive Residual Networks for Argument Mining](https://mlsanity.com/paper/2102.12227/) (2021)
- [Predicting Desirable Revisions of Evidence and Reasoning in Argumentative Writing](https://mlsanity.com/paper/2302.05039/) (2023)
- [A Hybrid Intelligence Method for Argument Mining](https://mlsanity.com/paper/2403.09713/) (2024)
- [Do You Hear The People Sing? Key Point Analysis via Iterative Clustering and Abstractive Summarisation](https://mlsanity.com/paper/2305.16000/) (2023)
- [Shaping Political Discourse using multi-source News Summarization](https://mlsanity.com/paper/2312.11703/) (2023)
- [Efficient argument classification with compact language models and ChatGPT-4 refinements](https://mlsanity.com/paper/2403.15473/) (2024)
- [Enhancing Argument Summarization: Prioritizing Exhaustiveness in Key Point Generation and Introducing an Automatic Coverage Evaluation Metric](https://mlsanity.com/paper/2404.11793/) (2024)
- [Limited Generalizability in Argument Mining: State-Of-The-Art Models Learn Datasets, Not Arguments](https://mlsanity.com/paper/2505.22137/) (2025)
- [The COVMis-Stance dataset: Stance Detection on Twitter for COVID-19 Misinformation](https://mlsanity.com/paper/2204.02000/) (2022)
- [How to Handle Different Types of Out-of-Distribution Scenarios in Computational Argumentation? A Comprehensive and Fine-Grained Field Study](https://mlsanity.com/paper/2309.08316/) (2023)
- [Argument Summarization and its Evaluation in the Era of Large Language Models](https://mlsanity.com/paper/2503.00847/) (2025)
- [ArgInstruct: Specialized Instruction Fine-Tuning for Computational Argumentation](https://mlsanity.com/paper/2505.22076/) (2025)

## Abstract (arXiv)

We experiment with two recent contextualized word embedding methods (ELMo and BERT) in the context of open-domain argument search. For the first time, we show how to leverage the power of contextualized word embeddings to classify and cluster topic-dependent arguments, achieving impressive results on both tasks and across multiple datasets. For argument classification, we improve the state-of-the-art for the UKP Sentential Argument Mining Corpus by 20.8 percentage points and for the IBM Debater - Evidence Sentences dataset by 7.4 percentage points. For the understudied task of argument clustering, we propose a pre-training step which improves by 7.8 percentage points over strong baselines on a novel dataset, and by 12.3 percentage points for the Argument Facet Similarity (AFS) Corpus.