# DYPLOC: Dynamic Planning of Content Using Mixed Language Models for Text Generation

arXiv: https://arxiv.org/abs/2106.00791 · HTML: https://mlsanity.com/paper/2106.00791/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 69 references that are in the map.
- First posted on arXiv 1 Jun 2021.

## Builds on (most cited first, 30 of 40)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [Recipes for building an open-domain chatbot](https://mlsanity.com/paper/2004.13637/) (2020)
- [The E2E Dataset: New Challenges For End-to-End Generation](https://mlsanity.com/paper/1706.09254/) (2017)
- [Plan-And-Write: Towards Better Automatic Storytelling](https://mlsanity.com/paper/1811.05701/) (2018)
- [Challenges in Data-to-Document Generation](https://mlsanity.com/paper/1707.08052/) (2017)
- [A Persona-Based Neural Conversation Model](https://mlsanity.com/paper/1603.06155/) (2016)
- [Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems](https://mlsanity.com/paper/1508.01745/) (2015)
- [Strategies for Structuring Story Generation](https://mlsanity.com/paper/1902.01109/) (2019)
- [Improving Conversational Recommender Systems via Knowledge Graph based Semantic Fusion](https://mlsanity.com/paper/2007.04032/) (2020)
- [Investigating Pretrained Language Models for Graph-to-Text Generation](https://mlsanity.com/paper/2007.08426/) (2020)
- [A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation](https://mlsanity.com/paper/2001.05139/) (2020)
- [Text-to-Text Pre-Training for Data-to-Text Tasks](https://mlsanity.com/paper/2005.10433/) (2020)
- [Content Planning for Neural Story Generation with Aristotelian Rescoring](https://mlsanity.com/paper/2009.09870/) (2020)
- [Step-by-Step: Separating Planning from Realization in Neural Data-to-Text Generation](https://mlsanity.com/paper/1904.03396/) (2019)
- [MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Models](https://mlsanity.com/paper/2010.00840/) (2020)
- [Unsupervised Opinion Summarization as Copycat-Review Generation](https://mlsanity.com/paper/1911.02247/) (2019)
- [Sticking to the Facts: Confident Decoding for Faithful Data-to-Text Generation](https://mlsanity.com/paper/1910.08684/) (2019)
- [Neural Syntactic Preordering for Controlled Paraphrase Generation](https://mlsanity.com/paper/2005.02013/) (2020)
- [Exploring Controllable Text Generation Techniques](https://mlsanity.com/paper/2005.01822/) (2020)
- [Neural data-to-text generation: A comparison between pipeline and end-to-end architectures](https://mlsanity.com/paper/1908.09022/) (2019)
- [A Controllable Model of Grounded Response Generation](https://mlsanity.com/paper/2005.00613/) (2020)
- [Neural Network-Based Abstract Generation for Opinions and Arguments](https://mlsanity.com/paper/1606.02785/) (2016)

## Built on by (most cited first, 11 of 11)

- [Pretrained Language Models for Text Generation: A Survey](https://mlsanity.com/paper/2201.05273/) (2022)
- [PLANET: Dynamic Content Planning in Autoregressive Transformers for Long-form Text Generation](https://mlsanity.com/paper/2203.09100/) (2022)
- [Dialogue Planning via Brownian Bridge Stochastic Process for Goal-directed Proactive Dialogue](https://mlsanity.com/paper/2305.05290/) (2023)
- [AMERICANO: Argument Generation with Discourse-driven Decomposition and Agent Interaction](https://mlsanity.com/paper/2310.20352/) (2023)
- [Target-constrained Bidirectional Planning for Generation of Target-oriented Proactive Dialogue](https://mlsanity.com/paper/2403.06063/) (2024)
- [MOCHA: A Multi-Task Training Approach for Coherent Text Generation from Cognitive Perspective](https://mlsanity.com/paper/2210.14650/) (2022)
- [Integrating AI Planning with Natural Language Processing: A Combination of Explicit and Tacit Knowledge](https://mlsanity.com/paper/2202.07138/) (2022)
- [RSTGen: Imbuing Fine-Grained Interpretable Control into Long-FormText Generators](https://mlsanity.com/paper/2205.12590/) (2022)
- [Text-Based Action-Model Acquisition for Planning](https://mlsanity.com/paper/2202.08373/) (2022)
- [Prove Your Point!: Bringing Proof-Enhancement Principles to Argumentative Essay Generation](https://mlsanity.com/paper/2410.22642/) (2024)
- [ARGUS: Theory-of-Mind Guided Argument Generation with Strategy-Aware Planning and Knowledge Grounding](https://mlsanity.com/paper/2608.20405/) (2026)

## Abstract (arXiv)

We study the task of long-form opinion text generation, which faces at least two distinct challenges. First, existing neural generation models fall short of coherence, thus requiring efficient content planning. Second, diverse types of information are needed to guide the generator to cover both subjective and objective content. To this end, we propose DYPLOC, a generation framework that conducts dynamic planning of content while generating the output based on a novel design of mixed language models. To enrich the generation with diverse content, we further propose to use large pre-trained models to predict relevant concepts and to generate claims. We experiment with two challenging tasks on newly collected datasets: (1) argument generation with Reddit ChangeMyView, and (2) writing articles using New York Times' Opinion section. Automatic evaluation shows that our model significantly outperforms competitive comparisons. Human judges further confirm that our generations are more coherent with richer content.