# Multi-Task Attentive Residual Networks for Argument Mining

arXiv: https://arxiv.org/abs/2102.12227 · HTML: https://mlsanity.com/paper/2102.12227/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 93 references that are in the map.
- First posted on arXiv 24 Feb 2021.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [A Primer in BERTology: What we know about how BERT works](https://mlsanity.com/paper/2002.12327/) (2020)
- [Attention is not not Explanation](https://mlsanity.com/paper/1908.04626/) (2019)
- [Green AI](https://mlsanity.com/paper/1907.10597/) (2019)
- [Parsing Argumentation Structures in Persuasive Essays](https://mlsanity.com/paper/1604.07370/) (2016)
- [Attention in Natural Language Processing](https://mlsanity.com/paper/1902.02181/) (2019)
- [Interactive Attention Networks for Aspect-Level Sentiment Classification](https://mlsanity.com/paper/1709.00893/) (2017)
- [Reporting Score Distributions Makes a Difference: Performance Study of LSTM-networks for Sequence Tagging](https://mlsanity.com/paper/1707.09861/) (2017)
- [Argumentation Mining in User-Generated Web Discourse](https://mlsanity.com/paper/1601.02403/) (2016)
- [Cross-topic Argument Mining from Heterogeneous Sources Using Attention-based Neural Networks](https://mlsanity.com/paper/1802.05758/) (2018)
- [Classification and Clustering of Arguments with Contextualized Word Embeddings](https://mlsanity.com/paper/1906.09821/) (2019)
- [Neural End-to-End Learning for Computational Argumentation Mining](https://mlsanity.com/paper/1704.06104/) (2017)
- [Recurrent Highway Networks](https://mlsanity.com/paper/1607.03474/) (2016)
- [Argument Mining with Structured SVMs and RNNs](https://mlsanity.com/paper/1704.06869/) (2017)
- [Corpus Wide Argument Mining -- a Working Solution](https://mlsanity.com/paper/1911.10763/) (2019)
- [Fine-Grained Argument Unit Recognition and Classification](https://mlsanity.com/paper/1904.09688/) (2019)

## Built on by (most cited first, 6 of 6)

- [Natural Language Processing for the Legal Domain: A Survey of Tasks, Datasets, Models, and Challenges](https://mlsanity.com/paper/2410.21306/) (2024)
- [Large Language Models in Argument Mining: A Survey](https://mlsanity.com/paper/2506.16383/) (2025)
- [Tree-Constrained Graph Neural Networks For Argument Mining](https://mlsanity.com/paper/2110.00124/) (2021)
- [Enhancing Legal Argument Mining with Domain Pre-training and Neural Networks](https://mlsanity.com/paper/2202.13457/) (2022)
- [Cross-Genre Argument Mining: Can Language Models Automatically Fill in Missing Discourse Markers?](https://mlsanity.com/paper/2306.04314/) (2023)
- [DMON: A Simple yet Effective Approach for Argument Structure Learning](https://mlsanity.com/paper/2405.01216/) (2024)

## Abstract (arXiv)

We explore the use of residual networks and neural attention for multiple argument mining tasks. We propose a residual architecture that exploits attention, multi-task learning, and makes use of ensemble, without any assumption on document or argument structure. We present an extensive experimental evaluation on five different corpora of user-generated comments, scientific publications, and persuasive essays. Our results show that our approach is a strong competitor against state-of-the-art architectures with a higher computational footprint or corpus-specific design, representing an interesting compromise between generality, performance accuracy and reduced model size.