# Efficient Argument Structure Extraction with Transfer Learning and Active Learning

arXiv: https://arxiv.org/abs/2204.00707 · HTML: https://mlsanity.com/paper/2204.00707/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 61 references that are in the map.
- First posted on arXiv 1 Apr 2022.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents](https://mlsanity.com/paper/1804.05685/) (2018)
- [Cold-start Active Learning through Self-supervised Language Modeling](https://mlsanity.com/paper/2010.09535/) (2020)
- [Deep Active Learning for Named Entity Recognition](https://mlsanity.com/paper/1707.05928/) (2017)
- [Parsing Argumentation Structures in Persuasive Essays](https://mlsanity.com/paper/1604.07370/) (2016)
- [Learning a Neural Semantic Parser from User Feedback](https://mlsanity.com/paper/1704.08760/) (2017)
- [Argumentation Mining in User-Generated Web Discourse](https://mlsanity.com/paper/1601.02403/) (2016)
- [Practical Obstacles to Deploying Active Learning](https://mlsanity.com/paper/1807.04801/) (2018)
- [Argument Mining for Understanding Peer Reviews](https://mlsanity.com/paper/1903.10104/) (2019)
- [AMPERSAND: Argument Mining for PERSuAsive oNline Discussions](https://mlsanity.com/paper/2004.14677/) (2020)
- [Argument Mining with Structured SVMs and RNNs](https://mlsanity.com/paper/1704.06869/) (2017)
- [Exploring the Role of Argument Structure in Online Debate Persuasion](https://mlsanity.com/paper/2010.03538/) (2020)
- [Analysis of Automatic Annotation Suggestions for Hard Discourse-Level Tasks in Expert Domains](https://mlsanity.com/paper/1906.02564/) (2019)
- [Dissecting Content and Context in Argumentative Relation Analysis](https://mlsanity.com/paper/1906.03338/) (2019)
- [Understanding and Detecting Supporting Arguments of Diverse Types](https://mlsanity.com/paper/1705.00045/) (2017)

## Built on by (most cited first, 5 of 5)

- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [A Survey on Deep Active Learning: Recent Advances and New Frontiers](https://mlsanity.com/paper/2405.00334/) (2024)
- [A Survey of Active Learning for Natural Language Processing](https://mlsanity.com/paper/2210.10109/) (2022)
- [Generative AI for Education (GAIED): Advances, Opportunities, and Challenges](https://mlsanity.com/paper/2402.01580/) (2024)
- [Enhancing Argument Structure Extraction with Efficient Leverage of Contextual Information](https://mlsanity.com/paper/2310.05073/) (2023)

## Abstract (arXiv)

The automation of extracting argument structures faces a pair of challenges on (1) encoding long-term contexts to facilitate comprehensive understanding, and (2) improving data efficiency since constructing high-quality argument structures is time-consuming. In this work, we propose a novel context-aware Transformer-based argument structure prediction model which, on five different domains, significantly outperforms models that rely on features or only encode limited contexts. To tackle the difficulty of data annotation, we examine two complementary methods: (i) transfer learning to leverage existing annotated data to boost model performance in a new target domain, and (ii) active learning to strategically identify a small amount of samples for annotation. We further propose model-independent sample acquisition strategies, which can be generalized to diverse domains. With extensive experiments, we show that our simple-yet-effective acquisition strategies yield competitive results against three strong comparisons. Combined with transfer learning, substantial F1 score boost (5-25) can be further achieved during the early iterations of active learning across domains.