# Examining the rhetorical capacities of neural language models

arXiv: https://arxiv.org/abs/2010.00153 · HTML: https://mlsanity.com/paper/2010.00153/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Oct 2020.

## Built on by (most cited first, 7 of 7)

- [Discourse Probing of Pretrained Language Models](https://mlsanity.com/paper/2104.05882/) (2021)
- [Linguistic Interpretability of Transformer-based Language Models: a systematic review](https://mlsanity.com/paper/2504.08001/) (2025)
- [Towards Understanding Large-Scale Discourse Structures in Pre-Trained and Fine-Tuned Language Models](https://mlsanity.com/paper/2204.04289/) (2022)
- [On the data requirements of probing](https://mlsanity.com/paper/2202.12801/) (2022)
- [Discourse Structure Extraction from Pre-Trained and Fine-Tuned Language Models in Dialogues](https://mlsanity.com/paper/2302.05895/) (2023)
- [Quantifying the Task-Specific Information in Text-Based Classifications](https://mlsanity.com/paper/2110.08931/) (2021)
- [Towards Domain-Independent Supervised Discourse Parsing Through Gradient Boosting](https://mlsanity.com/paper/2210.09565/) (2022)

## Abstract (arXiv)

Recently, neural language models (LMs) have demonstrated impressive abilities in generating high-quality discourse. While many recent papers have analyzed the syntactic aspects encoded in LMs, there has been no analysis to date of the inter-sentential, rhetorical knowledge. In this paper, we propose a method that quantitatively evaluates the rhetorical capacities of neural LMs. We examine the capacities of neural LMs understanding the rhetoric of discourse by evaluating their abilities to encode a set of linguistic features derived from Rhetorical Structure Theory (RST). Our experiments show that BERT-based LMs outperform other Transformer LMs, revealing the richer discourse knowledge in their intermediate layer representations. In addition, GPT-2 and XLNet apparently encode less rhetorical knowledge, and we suggest an explanation drawing from linguistic philosophy. Our method shows an avenue towards quantifying the rhetorical capacities of neural LMs.