# Controllable Natural Language Generation with Contrastive Prefixes

arXiv: https://arxiv.org/abs/2202.13257 · HTML: https://mlsanity.com/paper/2202.13257/

## Key facts

- Cited by 75 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 23 references that are in the map.
- First posted on arXiv 27 Feb 2022.

## Builds on (most cited first, 17 of 17)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://mlsanity.com/paper/2009.11462/) (2020)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [GeDi: Generative Discriminator Guided Sequence Generation](https://mlsanity.com/paper/2009.06367/) (2020)
- [FUDGE: Controlled Text Generation With Future Discriminators](https://mlsanity.com/paper/2104.05218/) (2021)
- [Toward Controlled Generation of Text](https://mlsanity.com/paper/1703.00955/) (2017)
- [Challenges in Detoxifying Language Models](https://mlsanity.com/paper/2109.07445/) (2021)
- [Controlling Linguistic Style Aspects in Neural Language Generation](https://mlsanity.com/paper/1707.02633/) (2017)
- [Learning to Write with Cooperative Discriminators](https://mlsanity.com/paper/1805.06087/) (2018)
- [Attribute Alignment: Controlling Text Generation from Pre-trained Language Models](https://mlsanity.com/paper/2103.11070/) (2021)

## Built on by (most cited first, 30 of 75)

- [Instruction Tuning for Large Language Models: A Survey](https://mlsanity.com/paper/2308.10792/) (2023)
- [RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning](https://mlsanity.com/paper/2205.12548/) (2022)
- [A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systems](https://mlsanity.com/paper/2402.18013/) (2024)
- [Large Language Models for Code: Security Hardening and Adversarial Testing](https://mlsanity.com/paper/2302.05319/) (2023)
- [Gradient-Based Constrained Sampling from Language Models](https://mlsanity.com/paper/2205.12558/) (2022)
- [Generative Knowledge Graph Construction: A Review](https://mlsanity.com/paper/2210.12714/) (2022)
- [Confidence Matters: Revisiting Intrinsic Self-Correction Capabilities of Large Language Models](https://mlsanity.com/paper/2402.12563/) (2024)
- [Critic-Guided Decoding for Controlled Text Generation](https://mlsanity.com/paper/2212.10938/) (2022)
- [A Distributional Lens for Multi-Aspect Controllable Text Generation](https://mlsanity.com/paper/2210.02889/) (2022)
- [Unified Detoxifying and Debiasing in Language Generation via Inference-time Adaptive Optimization](https://mlsanity.com/paper/2210.04492/) (2022)
- [DisCup: Discriminator Cooperative Unlikelihood Prompt-tuning for Controllable Text Generation](https://mlsanity.com/paper/2210.09551/) (2022)
- [Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applications](https://mlsanity.com/paper/2311.05876/) (2023)
- [Controllable Text Generation via Probability Density Estimation in the Latent Space](https://mlsanity.com/paper/2212.08307/) (2022)
- [Benchmarking Large Language Models on Controllable Generation under Diversified Instructions](https://mlsanity.com/paper/2401.00690/) (2024)
- [Mixture of Soft Prompts for Controllable Data Generation](https://mlsanity.com/paper/2303.01580/) (2023)
- [Tailor: A Prompt-Based Approach to Attribute-Based Controlled Text Generation](https://mlsanity.com/paper/2204.13362/) (2022)
- [PromptEHR: Conditional Electronic Healthcare Records Generation with Prompt Learning](https://mlsanity.com/paper/2211.01761/) (2022)
- [Personalized Text Generation with Fine-Grained Linguistic Control](https://mlsanity.com/paper/2402.04914/) (2024)
- [From Google Gemini to OpenAI Q* (Q-Star): A Survey of Reshaping the Generative Artificial Intelligence (AI) Research Landscape](https://mlsanity.com/paper/2312.10868/) (2023)
- [Contrastive Instruction Tuning](https://mlsanity.com/paper/2402.11138/) (2024)
- [Joint Visual and Text Prompting for Improved Object-Centric Perception with Multimodal Large Language Models](https://mlsanity.com/paper/2404.04514/) (2024)
- [Walking in Others' Shoes: How Perspective-Taking Guides Large Language Models in Reducing Toxicity and Bias](https://mlsanity.com/paper/2407.15366/) (2024)
- [Controlled Text Generation for Large Language Model with Dynamic Attribute Graphs](https://mlsanity.com/paper/2402.11218/) (2024)
- [Reinforcement Learning with Token-level Feedback for Controllable Text Generation](https://mlsanity.com/paper/2403.11558/) (2024)
- [LLM-based multi-agent poetry generation in non-cooperative environments](https://mlsanity.com/paper/2409.03659/) (2024)
- [Dynamic Multi-Reward Weighting for Multi-Style Controllable Generation](https://mlsanity.com/paper/2402.14146/) (2024)
- [MacLaSa: Multi-Aspect Controllable Text Generation via Efficient Sampling from Compact Latent Space](https://mlsanity.com/paper/2305.12785/) (2023)
- [SimOAP: Improve Coherence and Consistency in Persona-based Dialogue Generation via Over-sampling and Post-evaluation](https://mlsanity.com/paper/2305.11130/) (2023)
- [An Extensible Plug-and-Play Method for Multi-Aspect Controllable Text Generation](https://mlsanity.com/paper/2212.09387/) (2022)
- [Air-Decoding: Attribute Distribution Reconstruction for Decoding-Time Controllable Text Generation](https://mlsanity.com/paper/2310.14892/) (2023)

## Abstract (arXiv)

To guide the generation of large pretrained language models (LM), previous work has focused on directly fine-tuning the language model or utilizing an attribute discriminator. In this work, we propose a novel lightweight framework for controllable GPT2 generation, which utilizes a set of small attribute-specific vectors, called prefixes, to steer natural language generation. Different from prefix-tuning, where each prefix is trained independently, we take the relationship among prefixes into consideration and train multiple prefixes simultaneously. We propose a novel supervised method and also an unsupervised method to train the prefixes for single-aspect control while the combination of these two methods can achieve multi-aspect control. Experimental results on both single-aspect and multi-aspect control show that our methods can guide generation towards the desired attributes while keeping high linguistic quality.