# AMELIA: A Family of Multi-task End-to-end Language Models for Argumentation

arXiv: https://arxiv.org/abs/2508.17926 · HTML: https://mlsanity.com/paper/2508.17926/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 85 references that are in the map.
- First posted on arXiv 25 Aug 2025.

## Builds on (most cited first, 30 of 38)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [Editing Models with Task Arithmetic](https://mlsanity.com/paper/2212.04089/) (2022)
- [TIES-Merging: Resolving Interference When Merging Models](https://mlsanity.com/paper/2306.01708/) (2023)
- [Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch](https://mlsanity.com/paper/2311.03099/) (2023)
- [Arcee's MergeKit: A Toolkit for Merging Large Language Models](https://mlsanity.com/paper/2403.13257/) (2024)
- [Parsing Argumentation Structures in Persuasive Essays](https://mlsanity.com/paper/1604.07370/) (2016)
- [DELLA-Merging: Reducing Interference in Model Merging through Magnitude-Based Sampling](https://mlsanity.com/paper/2406.11617/) (2024)
- [Thread of Thought Unraveling Chaotic Contexts](https://mlsanity.com/paper/2311.08734/) (2023)
- [Beyond Chain-of-Thought, Effective Graph-of-Thought Reasoning in Language Models](https://mlsanity.com/paper/2305.16582/) (2023)
- [Exploring the Potential of Large Language Models in Computational Argumentation](https://mlsanity.com/paper/2311.09022/) (2023)
- [Argument Mining with Structured SVMs and RNNs](https://mlsanity.com/paper/1704.06869/) (2017)
- [Are Large Language Models Reliable Argument Quality Annotators?](https://mlsanity.com/paper/2404.09696/) (2024)
- [IAM: A Comprehensive and Large-Scale Dataset for Integrated Argument Mining Tasks](https://mlsanity.com/paper/2203.12257/) (2022)
- [Can Large Language Models perform Relation-based Argument Mining?](https://mlsanity.com/paper/2402.11243/) (2024)
- [Classifying Argumentative Relations Using Logical Mechanisms and Argumentation Schemes](https://mlsanity.com/paper/2105.07571/) (2021)
- [Learning From Revisions: Quality Assessment of Claims in Argumentation at Scale](https://mlsanity.com/paper/2101.10250/) (2021)
- [Unsupervised Expressive Rules Provide Explainability and Assist Human Experts Grasping New Domains](https://mlsanity.com/paper/2010.09459/) (2020)
- [Which Side Are You On? A Multi-task Dataset for End-to-End Argument Summarisation and Evaluation](https://mlsanity.com/paper/2406.03151/) (2024)
- [Multi-Task Learning for Argumentation Mining in Low-Resource Settings](https://mlsanity.com/paper/1804.04083/) (2018)
- [AQE: Argument Quadruplet Extraction via a Quad-Tagging Augmented Generative Approach](https://mlsanity.com/paper/2305.19902/) (2023)

## Built on by (most cited first, 3 of 3)

- [Evaluating LLM-Driven Summarisation of Parliamentary Debates with Computational Argumentation](https://mlsanity.com/paper/2604.19331/) (2026)
- [From Argument Components to Graphs: A Multi-Agent Debate with Confidence Gating for Argument Relations](https://mlsanity.com/paper/2606.16047/) (2026)
- [Swiss4Arg: A Parallel Dataset for Multilingual Argument Mining across Swiss National Languages](https://mlsanity.com/paper/2610.17894/) (2026)

## Abstract (arXiv)

Argument mining is a subfield of argumentation that aims to automatically extract argumentative structures and their relations from natural language texts. This paper investigates how a single large language model can be leveraged to perform one or several argument mining tasks. Our contributions are two-fold. First, we construct a multi-task dataset by surveying and converting 19 well-known argument mining datasets from the literature into a unified format. Second, we explore various training strategies using Meta AI's Llama-3.1-8B-Instruct model: (1) fine-tuning on individual tasks, (2) fine-tuning jointly on multiple tasks, and (3) merging models fine-tuned separately on individual tasks. Our experiments show that task-specific fine-tuning significantly improves individual performance across all tasks. Moreover, multi-task fine-tuning maintains strong performance without degradation, suggesting effective transfer learning across related tasks. Finally, we demonstrate that model merging offers a viable compromise: it yields competitive performance while mitigating the computational costs associated with full multi-task fine-tuning.