# RSTGen: Imbuing Fine-Grained Interpretable Control into Long-FormText Generators

arXiv: https://arxiv.org/abs/2205.12590 · HTML: https://mlsanity.com/paper/2205.12590/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 42 references that are in the map.
- First posted on arXiv 25 May 2022.

## Builds on (most cited first, 30 of 30)

- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [Toward Controlled Generation of Text](https://mlsanity.com/paper/1703.00955/) (2017)
- [Plan-And-Write: Towards Better Automatic Storytelling](https://mlsanity.com/paper/1811.05701/) (2018)
- [Challenges in Data-to-Document Generation](https://mlsanity.com/paper/1707.08052/) (2017)
- [Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems](https://mlsanity.com/paper/1508.01745/) (2015)
- [Strategies for Structuring Story Generation](https://mlsanity.com/paper/1902.01109/) (2019)
- [Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks](https://mlsanity.com/paper/1810.09536/) (2018)
- [Step-by-Step: Separating Planning from Realization in Neural Data-to-Text Generation](https://mlsanity.com/paper/1904.03396/) (2019)
- [ConvoKit: A Toolkit for the Analysis of Conversations](https://mlsanity.com/paper/2005.04246/) (2020)
- [Tree Transformer: Integrating Tree Structures into Self-Attention](https://mlsanity.com/paper/1909.06639/) (2019)
- [Jointly Measuring Diversity and Quality in Text Generation Models](https://mlsanity.com/paper/1904.03971/) (2019)
- [Neural Syntactic Preordering for Controlled Paraphrase Generation](https://mlsanity.com/paper/2005.02013/) (2020)
- [Rethinking Action Spaces for Reinforcement Learning in End-to-end Dialog Agents with Latent Variable Models](https://mlsanity.com/paper/1902.08858/) (2019)
- [Topic-Guided Variational Autoencoders for Text Generation](https://mlsanity.com/paper/1903.07137/) (2019)
- [A Multi-Task Approach for Disentangling Syntax and Semantics in Sentence Representations](https://mlsanity.com/paper/1904.01173/) (2019)
- [GRUEN for Evaluating Linguistic Quality of Generated Text](https://mlsanity.com/paper/2010.02498/) (2020)
- [Constrained Decoding for Neural NLG from Compositional Representations in Task-Oriented Dialogue](https://mlsanity.com/paper/1906.07220/) (2019)
- [PAIR: Planning and Iterative Refinement in Pre-trained Transformers for Long Text Generation](https://mlsanity.com/paper/2010.02301/) (2020)
- [Tree-structured Attention with Hierarchical Accumulation](https://mlsanity.com/paper/2002.08046/) (2020)
- [Sequence-to-Sequence Generation for Spoken Dialogue via Deep Syntax Trees and Strings](https://mlsanity.com/paper/1606.05491/) (2016)
- [Sentence-Level Content Planning and Style Specification for Neural Text Generation](https://mlsanity.com/paper/1909.00734/) (2019)
- [Toward Fast and Accurate Neural Discourse Segmentation](https://mlsanity.com/paper/1808.09147/) (2018)
- [DiscoDVT: Generating Long Text with Discourse-Aware Discrete Variational Transformer](https://mlsanity.com/paper/2110.05999/) (2021)
- [DYPLOC: Dynamic Planning of Content Using Mixed Language Models for Text Generation](https://mlsanity.com/paper/2106.00791/) (2021)
- [Exploiting Syntactic Structure for Better Language Modeling: A Syntactic Distance Approach](https://mlsanity.com/paper/2005.05864/) (2020)
- [Transformer-Based Neural Text Generation with Syntactic Guidance](https://mlsanity.com/paper/2010.01737/) (2020)

## Built on by (most cited first, 3 of 3)

- [Disco-RAG: Discourse-Aware Retrieval-Augmented Generation](https://mlsanity.com/paper/2601.04377/) (2026)
- [Modeling Unified Semantic Discourse Structure for High-quality Headline Generation](https://mlsanity.com/paper/2403.15776/) (2024)
- [Writing Like the Best: Exemplar-Based Expository Text Generation](https://mlsanity.com/paper/2505.18859/) (2025)

## Abstract (arXiv)

In this paper, we study the task of improving the cohesion and coherence of long-form text generated by language models. To this end, we propose RSTGen, a framework that utilises Rhetorical Structure Theory (RST), a classical language theory, to control the discourse structure, semantics and topics of generated text. Firstly, we demonstrate our model's ability to control structural discourse and semantic features of generated text in open generation evaluation. Then we experiment on the two challenging long-form text tasks of argument generation and story generation. Evaluation using automated metrics and a metric with high correlation to human evaluation, shows that our model performs competitively against existing models, while offering significantly more controls over generated text than alternative methods.