# PCFG-based Natural Language Interface Improves Generalization for Controlled Text Generation

arXiv: https://arxiv.org/abs/2210.07431 · HTML: https://mlsanity.com/paper/2210.07431/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 37 references that are in the map.
- First posted on arXiv 14 Oct 2022.

## Builds on (most cited first, 28 of 28)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [Diffusion-LM Improves Controllable Text Generation](https://mlsanity.com/paper/2205.14217/) (2022)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [Cross-Task Generalization via Natural Language Crowdsourcing Instructions](https://mlsanity.com/paper/2104.08773/) (2021)
- [Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model](https://mlsanity.com/paper/2201.11990/) (2022)
- [DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts](https://mlsanity.com/paper/2105.03023/) (2021)
- [GeDi: Generative Discriminator Guided Sequence Generation](https://mlsanity.com/paper/2009.06367/) (2020)
- [FUDGE: Controlled Text Generation With Future Discriminators](https://mlsanity.com/paper/2104.05218/) (2021)
- [Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation](https://mlsanity.com/paper/1804.06609/) (2018)
- [COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics](https://mlsanity.com/paper/2202.11705/) (2022)
- [Generating Informative and Diverse Conversational Responses via Adversarial Information Maximization](https://mlsanity.com/paper/1809.05972/) (2018)
- [A Recipe For Arbitrary Text Style Transfer with Large Language Models](https://mlsanity.com/paper/2109.03910/) (2021)
- [Few-Shot Text Generation with Pattern-Exploiting Training](https://mlsanity.com/paper/2012.11926/) (2020)
- [A Plug-and-Play Method for Controlled Text Generation](https://mlsanity.com/paper/2109.09707/) (2021)
- [News Category Dataset](https://mlsanity.com/paper/2209.11429/) (2022)
- [Attribute Alignment: Controlling Text Generation from Pre-trained Language Models](https://mlsanity.com/paper/2103.11070/) (2021)
- [Constrained Abstractive Summarization: Preserving Factual Consistency with Constrained Generation](https://mlsanity.com/paper/2010.12723/) (2020)
- [Control Prefixes for Parameter-Efficient Text Generation](https://mlsanity.com/paper/2110.08329/) (2021)
- [Tailor: A Prompt-Based Approach to Attribute-Based Controlled Text Generation](https://mlsanity.com/paper/2204.13362/) (2022)
- [COD3S: Diverse Generation with Discrete Semantic Signatures](https://mlsanity.com/paper/2010.02882/) (2020)

## Built on by (most cited first, 4 of 4)

- [Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements](https://mlsanity.com/paper/2410.08968/) (2024)
- [Re-Imagining Multimodal Instruction Tuning: A Representation View](https://mlsanity.com/paper/2503.00723/) (2025)
- [Frame Representation Hypothesis: Multi-Token LLM Interpretability and Concept-Guided Text Generation](https://mlsanity.com/paper/2412.07334/) (2024)
- [HiCaM: A Hierarchical-Causal Modification Framework for Long-Form Text Modification](https://mlsanity.com/paper/2505.24319/) (2025)

## Abstract (arXiv)

Existing work on controlled text generation (CTG) assumes a control interface of categorical attributes. In this work, we propose a natural language (NL) interface, where we craft a PCFG to embed the control attributes into natural language commands, and propose variants of existing CTG models that take commands as input. In our experiments, we design tailored setups to test model's generalization abilities. We find our PCFG-based command generation approach is effective for handling unseen commands compared to fix-set templates; our proposed NL models can effectively generalize to unseen attributes, a new ability enabled by the NL interface, as well as unseen attribute combinations. Interestingly, we discover that the simple conditional generation approach, enhanced with our proposed NL interface, is a strong baseline in those challenging settings.