# Style Control for Schema-Guided Natural Language Generation

arXiv: https://arxiv.org/abs/2109.12211 · HTML: https://mlsanity.com/paper/2109.12211/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 51 references that are in the map.
- First posted on arXiv 24 Sep 2021.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [The Woman Worked as a Babysitter: On Biases in Language Generation](https://mlsanity.com/paper/1909.01326/) (2019)
- [DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts](https://mlsanity.com/paper/2105.03023/) (2021)
- [Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset](https://mlsanity.com/paper/1909.05855/) (2019)
- [GeDi: Generative Discriminator Guided Sequence Generation](https://mlsanity.com/paper/2009.06367/) (2020)
- [SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient](https://mlsanity.com/paper/1609.05473/) (2016)
- [Toward Controlled Generation of Text](https://mlsanity.com/paper/1703.00955/) (2017)
- [Why We Need New Evaluation Metrics for NLG](https://mlsanity.com/paper/1707.06875/) (2017)
- [Controlling Linguistic Style Aspects in Neural Language Generation](https://mlsanity.com/paper/1707.02633/) (2017)
- [What makes a good conversation? How controllable attributes affect human judgments](https://mlsanity.com/paper/1902.08654/) (2019)
- [Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems](https://mlsanity.com/paper/1508.01745/) (2015)
- [Learning to Write with Cooperative Discriminators](https://mlsanity.com/paper/1805.06087/) (2018)
- [MIME: MIMicking Emotions for Empathetic Response Generation](https://mlsanity.com/paper/2010.01454/) (2020)
- [Controllable Abstractive Summarization](https://mlsanity.com/paper/1711.05217/) (2017)
- [Hello, It's GPT-2 -- How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1907.05774/) (2019)
- [MoEL: Mixture of Empathetic Listeners](https://mlsanity.com/paper/1908.07687/) (2019)
- [Few-shot Natural Language Generation for Task-Oriented Dialog](https://mlsanity.com/paper/2002.12328/) (2020)
- [Controlling Output Length in Neural Encoder-Decoders](https://mlsanity.com/paper/1609.09552/) (2016)
- [Step-by-Step: Separating Planning from Realization in Neural Data-to-Text Generation](https://mlsanity.com/paper/1904.03396/) (2019)
- [MojiTalk: Generating Emotional Responses at Scale](https://mlsanity.com/paper/1711.04090/) (2017)
- [Leveraging Slot Descriptions for Zero-Shot Cross-Domain Dialogue State Tracking](https://mlsanity.com/paper/2105.04222/) (2021)
- [Modeling Empathy and Distress in Reaction to News Stories](https://mlsanity.com/paper/1808.10399/) (2018)
- [Adding Chit-Chat to Enhance Task-Oriented Dialogues](https://mlsanity.com/paper/2010.12757/) (2020)
- [Towards Zero-Shot Frame Semantic Parsing for Domain Scaling](https://mlsanity.com/paper/1707.02363/) (2017)
- [Findings of the E2E NLG Challenge](https://mlsanity.com/paper/1810.01170/) (2018)
- [Controlling Personality-Based Stylistic Variation with Neural Natural Language Generators](https://mlsanity.com/paper/1805.08352/) (2018)

## Built on by (most cited first, 10 of 10)

- [FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Models](https://mlsanity.com/paper/2310.20410/) (2023)
- [CFBench: A Comprehensive Constraints-Following Benchmark for LLMs](https://mlsanity.com/paper/2408.01122/) (2024)
- [Controlling Personality Style in Dialogue with Zero-Shot Prompt-Based Learning](https://mlsanity.com/paper/2302.03848/) (2023)
- ["What makes a question inquisitive?" A Study on Type-Controlled Inquisitive Question Generation](https://mlsanity.com/paper/2205.08056/) (2022)
- [Controlled Language Generation for Language Learning Items](https://mlsanity.com/paper/2211.15731/) (2022)
- [A Critical Reflection and Forward Perspective on Empathy and Natural Language Processing](https://mlsanity.com/paper/2210.16604/) (2022)
- [Conversation Style Transfer using Few-Shot Learning](https://mlsanity.com/paper/2302.08362/) (2023)
- [Schema-Guided Semantic Accuracy: Faithfulness in Task-Oriented Dialogue Response Generation](https://mlsanity.com/paper/2301.12568/) (2023)
- [Unsupervised Neural Stylistic Text Generation using Transfer learning and Adapters](https://mlsanity.com/paper/2210.03264/) (2022)
- [Authorship Impersonation via LLM Prompting does not Evade Authorship Verification Methods](https://mlsanity.com/paper/2603.29454/) (2026)

## Abstract (arXiv)

Natural Language Generation (NLG) for task-oriented dialogue systems focuses on communicating specific content accurately, fluently, and coherently. While these attributes are crucial for a successful dialogue, it is also desirable to simultaneously accomplish specific stylistic goals, such as response length, point-of-view, descriptiveness, sentiment, formality, and empathy. In this work, we focus on stylistic control and evaluation for schema-guided NLG, with joint goals of achieving both semantic and stylistic control. We experiment in detail with various controlled generation methods for large pretrained language models: specifically, conditional training, guided fine-tuning, and guided decoding. We discuss their advantages and limitations, and evaluate them with a broad range of automatic and human evaluation metrics. Our results show that while high style accuracy and semantic correctness are easier to achieve for more lexically-defined styles with conditional training, stylistic control is also achievable for more semantically complex styles using discriminator-based guided decoding methods. The results also suggest that methods that are more scalable (with less hyper-parameters tuning) and that disentangle content generation and stylistic variations are more effective at achieving semantic correctness and style accuracy.