# Can we obtain significant success in RST discourse parsing by using Large Language Models?

arXiv: https://arxiv.org/abs/2403.05065 · HTML: https://mlsanity.com/paper/2403.05065/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 48 references that are in the map.
- First posted on arXiv 8 Mar 2024.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [Discourse-Aware Neural Extractive Text Summarization](https://mlsanity.com/paper/1910.14142/) (2019)
- [Discern: Discourse-Aware Entailment Reasoning Network for Conversational Machine Reading](https://mlsanity.com/paper/2010.01838/) (2020)
- [Exploring the Trade-Offs: Unified Large Language Models vs Local Fine-Tuned Models for Highly-Specific Radiology NLI Task](https://mlsanity.com/paper/2304.09138/) (2023)
- [DMRST: A Joint Framework for Document-Level Multilingual RST Discourse Segmentation and Parsing](https://mlsanity.com/paper/2110.04518/) (2021)
- [Better Document-level Sentiment Analysis from RST Discourse Parsing](https://mlsanity.com/paper/1509.01599/) (2015)
- [Polyglot: Distributed Word Representations for Multilingual NLP](https://mlsanity.com/paper/1307.1662/) (2013)
- [A Unified Linear-Time Framework for Sentence-Level Discourse Parsing](https://mlsanity.com/paper/1905.05682/) (2019)
- [Why Can't Discourse Parsing Generalize? A Thorough Investigation of the Impact of Data Diversity](https://mlsanity.com/paper/2302.06488/) (2023)
- [RST Parsing from Scratch](https://mlsanity.com/paper/2105.10861/) (2021)
- [Modeling Discourse Structure for Document-level Neural Machine Translation](https://mlsanity.com/paper/2006.04721/) (2020)
- [Discourse Structure in Machine Translation Evaluation](https://mlsanity.com/paper/1710.01504/) (2017)
- [A Simple and Strong Baseline for End-to-End Neural RST-style Discourse Parsing](https://mlsanity.com/paper/2210.08355/) (2022)
- [An End-to-End Document-Level Neural Discourse Parser Exploiting Multi-Granularity Representations](https://mlsanity.com/paper/2012.11169/) (2020)

## Built on by (most cited first, 4 of 4)

- [Llamipa: An Incremental Discourse Parser](https://mlsanity.com/paper/2406.18256/) (2024)
- [Document Understanding, Measurement, and Manipulation Using Category Theory](https://mlsanity.com/paper/2510.21553/) (2025)
- [On the Influence of Discourse Relations in Persuasive Texts](https://mlsanity.com/paper/2510.26124/) (2025)
- [MGAL: A Multilingual Granularity-Aware Long-Context Benchmark](https://mlsanity.com/paper/2608.20853/) (2026)

## Abstract (arXiv)

Recently, decoder-only pre-trained large language models (LLMs), with several tens of billion parameters, have significantly impacted a wide range of natural language processing (NLP) tasks. While encoder-only or encoder-decoder pre-trained language models have already proved to be effective in discourse parsing, the extent to which LLMs can perform this task remains an open research question. Therefore, this paper explores how beneficial such LLMs are for Rhetorical Structure Theory (RST) discourse parsing. Here, the parsing process for both fundamental top-down and bottom-up strategies is converted into prompts, which LLMs can work with. We employ Llama 2 and fine-tune it with QLoRA, which has fewer parameters that can be tuned. Experimental results on three benchmark datasets, RST-DT, Instr-DT, and the GUM corpus, demonstrate that Llama 2 with 70 billion parameters in the bottom-up strategy obtained state-of-the-art (SOTA) results with significant differences. Furthermore, our parsers demonstrated generalizability when evaluated on RST-DT, showing that, in spite of being trained with the GUM corpus, it obtained similar performances to those of existing parsers trained with RST-DT.