# Can Unsupervised Knowledge Transfer from Social Discussions Help Argument Mining?

arXiv: https://arxiv.org/abs/2203.12881 · HTML: https://mlsanity.com/paper/2203.12881/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 33 references that are in the map.
- First posted on arXiv 24 Mar 2022.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [AllenNLP: A Deep Semantic Natural Language Processing Platform](https://mlsanity.com/paper/1803.07640/) (2018)
- [On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines](https://mlsanity.com/paper/2006.04884/) (2020)
- [Winning Arguments: Interaction Dynamics and Persuasion Strategies in Good-faith Online Discussions](https://mlsanity.com/paper/1602.01103/) (2016)
- [Parsing Argumentation Structures in Persuasive Essays](https://mlsanity.com/paper/1604.07370/) (2016)
- [Reporting Score Distributions Makes a Difference: Performance Study of LSTM-networks for Sequence Tagging](https://mlsanity.com/paper/1707.09861/) (2017)
- [ConvoKit: A Toolkit for the Analysis of Conversations](https://mlsanity.com/paper/2005.04246/) (2020)
- [A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks](https://mlsanity.com/paper/2010.03648/) (2020)
- [Argumentation Mining in User-Generated Web Discourse](https://mlsanity.com/paper/1601.02403/) (2016)
- [Neural End-to-End Learning for Computational Argumentation Mining](https://mlsanity.com/paper/1704.06104/) (2017)
- [AMPERSAND: Argument Mining for PERSuAsive oNline Discussions](https://mlsanity.com/paper/2004.14677/) (2020)
- [Adaptive Self-training for Few-shot Neural Sequence Labeling](https://mlsanity.com/paper/2010.03680/) (2020)
- [Changing Views: Persuasion Modeling and Argument Extraction from Online Discussions](https://mlsanity.com/paper/1907.06076/) (2019)
- [Multi-Task Learning for Argumentation Mining in Low-Resource Settings](https://mlsanity.com/paper/1804.04083/) (2018)
- [Dissecting Content and Context in Argumentative Relation Analysis](https://mlsanity.com/paper/1906.03338/) (2019)
- [The evolution of argumentation mining: From models to social media and emerging tools](https://mlsanity.com/paper/1907.02258/) (2019)

## Built on by (most cited first, 5 of 5)

- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [Social Influence Dialogue Systems: A Survey of Datasets and Models For Social Influence Tasks](https://mlsanity.com/paper/2210.05664/) (2022)
- [Cross-Genre Argument Mining: Can Language Models Automatically Fill in Missing Discourse Markers?](https://mlsanity.com/paper/2306.04314/) (2023)
- [Opinion Mining Using Population-tuned Generative Language Models](https://mlsanity.com/paper/2307.13173/) (2023)
- [Exploration of Marker-Based Approaches in Argument Mining through Augmented Natural Language](https://mlsanity.com/paper/2406.08606/) (2024)

## Abstract (arXiv)

Identifying argument components from unstructured texts and predicting the relationships expressed among them are two primary steps of argument mining. The intrinsic complexity of these tasks demands powerful learning models. While pretrained Transformer-based Language Models (LM) have been shown to provide state-of-the-art results over different NLP tasks, the scarcity of manually annotated data and the highly domain-dependent nature of argumentation restrict the capabilities of such models. In this work, we propose a novel transfer learning strategy to overcome these challenges. We utilize argumentation-rich social discussions from the ChangeMyView subreddit as a source of unsupervised, argumentative discourse-aware knowledge by finetuning pretrained LMs on a selectively masked language modeling task. Furthermore, we introduce a novel prompt-based strategy for inter-component relation prediction that compliments our proposed finetuning method while leveraging on the discourse context. Exhaustive experiments show the generalization capability of our method on these two tasks over within-domain as well as out-of-domain datasets, outperforming several existing and employed strong baselines.