# Cross-Genre Argument Mining: Can Language Models Automatically Fill in Missing Discourse Markers?

arXiv: https://arxiv.org/abs/2306.04314 · HTML: https://mlsanity.com/paper/2306.04314/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 78 references that are in the map.
- First posted on arXiv 7 Jun 2023.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Parsing Argumentation Structures in Persuasive Essays](https://mlsanity.com/paper/1604.07370/) (2016)
- [ChatGPT: A Meta-Analysis after 2.5 Months](https://mlsanity.com/paper/2302.13795/) (2023)
- [Neural End-to-End Learning for Computational Argumentation Mining](https://mlsanity.com/paper/1704.06104/) (2017)
- [Mining Legal Arguments in Court Decisions](https://mlsanity.com/paper/2208.06178/) (2022)
- [Mining Discourse Markers for Unsupervised Sentence Representation Learning](https://mlsanity.com/paper/1903.11850/) (2019)
- [Multilingual Argument Mining: Datasets and Analysis](https://mlsanity.com/paper/2010.06432/) (2020)
- [Specialising Word Vectors for Lexical Entailment](https://mlsanity.com/paper/1710.06371/) (2017)
- [Adversarial Connective-exploiting Networks for Implicit Discourse Relation Classification](https://mlsanity.com/paper/1704.00217/) (2017)
- [Cross-lingual Argumentation Mining: Machine Translation (and a bit of Projection) is All You Need!](https://mlsanity.com/paper/1807.08998/) (2018)
- [Learning to Explicitate Connectives with Seq2Seq Network for Implicit Discourse Relation Classification](https://mlsanity.com/paper/1811.01697/) (2018)
- [Multi-Task Learning for Argumentation Mining in Low-Resource Settings](https://mlsanity.com/paper/1804.04083/) (2018)
- [Dissecting Content and Context in Argumentative Relation Analysis](https://mlsanity.com/paper/1906.03338/) (2019)
- [Building a Web-Scale Dependency-Parsed Corpus from CommonCrawl](https://mlsanity.com/paper/1710.01779/) (2017)
- [Let's be explicit about that: Distant supervision for implicit discourse relation classification via connective prediction](https://mlsanity.com/paper/2106.03192/) (2021)
- [Argument Invention from First Principles](https://mlsanity.com/paper/1908.08336/) (2019)
- [Using Argument-based Features to Predict and Analyse Review Helpfulness](https://mlsanity.com/paper/1707.07279/) (2017)
- [Argumentative Relation Classification as Plausibility Ranking](https://mlsanity.com/paper/1909.09031/) (2019)
- [Multi-Task Attentive Residual Networks for Argument Mining](https://mlsanity.com/paper/2102.12227/) (2021)
- [Can Unsupervised Knowledge Transfer from Social Discussions Help Argument Mining?](https://mlsanity.com/paper/2203.12881/) (2022)
- [Discourse Marker Augmented Network with Reinforcement Learning for Natural Language Inference](https://mlsanity.com/paper/1907.09692/) (2019)
- [Unsupervised Adversarial Domain Adaptation for Implicit Discourse Relation Classification](https://mlsanity.com/paper/2003.02244/) (2020)
- [Multi-Task and Multi-Corpora Training Strategies to Enhance Argumentative Sentence Linking Performance](https://mlsanity.com/paper/2109.13067/) (2021)

## Built on by (most cited first, 1 of 1)

- [Can Large Language Models Handle Discourse Particles? A Case Study of Colloquial Malay](https://mlsanity.com/paper/2605.28782/) (2026)

## Abstract (arXiv)

Available corpora for Argument Mining differ along several axes, and one of the key differences is the presence (or absence) of discourse markers to signal argumentative content. Exploring effective ways to use discourse markers has received wide attention in various discourse parsing tasks, from which it is well-known that discourse markers are strong indicators of discourse relations. To improve the robustness of Argument Mining systems across different genres, we propose to automatically augment a given text with discourse markers such that all relations are explicitly signaled. Our analysis unveils that popular language models taken out-of-the-box fail on this task; however, when fine-tuned on a new heterogeneous dataset that we construct (including synthetic and real examples), they perform considerably better. We demonstrate the impact of our approach on an Argument Mining downstream task, evaluated on different corpora, showing that language models can be trained to automatically fill in discourse markers across different corpora, improving the performance of a downstream model in some, but not all, cases. Our proposed approach can further be employed as an assistive tool for better discourse understanding.