# A Causal Lens for Controllable Text Generation

arXiv: https://arxiv.org/abs/2201.09119 · HTML: https://mlsanity.com/paper/2201.09119/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 60 of its 89 references that are in the map.
- First posted on arXiv 22 Jan 2022.

## Builds on (most cited first, 30 of 60)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [Universal Adversarial Triggers for Attacking and Analyzing NLP](https://mlsanity.com/paper/1908.07125/) (2019)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [The Woman Worked as a Babysitter: On Biases in Language Generation](https://mlsanity.com/paper/1909.01326/) (2019)
- [Learning the Difference that Makes a Difference with Counterfactually-Augmented Data](https://mlsanity.com/paper/1909.12434/) (2019)
- [Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints](https://mlsanity.com/paper/1707.09457/) (2017)
- [Causal Effect Inference with Deep Latent-Variable Models](https://mlsanity.com/paper/1705.08821/) (2017)
- [GeDi: Generative Discriminator Guided Sequence Generation](https://mlsanity.com/paper/2009.06367/) (2020)
- [Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting](https://mlsanity.com/paper/1901.09451/) (2019)
- [Learning Representations for Counterfactual Inference](https://mlsanity.com/paper/1605.03661/) (2016)
- [Toward Controlled Generation of Text](https://mlsanity.com/paper/1703.00955/) (2017)
- [Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation](https://mlsanity.com/paper/1911.11834/) (2019)
- [Evaluating Gender Bias in Machine Translation](https://mlsanity.com/paper/1906.00591/) (2019)
- [Style Transfer from Non-Parallel Text by Cross-Alignment](https://mlsanity.com/paper/1705.09655/) (2017)
- [Causality for Machine Learning](https://mlsanity.com/paper/1911.10500/) (2019)
- [AttGAN: Facial Attribute Editing by Only Changing What You Want](https://mlsanity.com/paper/1711.10678/) (2017)
- [Causal Intervention for Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2009.12547/) (2020)
- [Deep Structural Causal Models for Tractable Counterfactual Inference](https://mlsanity.com/paper/2006.06485/) (2020)
- [A Survey on Causal Inference](https://mlsanity.com/paper/2002.02770/) (2020)
- [Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology](https://mlsanity.com/paper/1906.04571/) (2019)
- [Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer](https://mlsanity.com/paper/1804.06437/) (2018)
- [Deep Learning for Text Style Transfer: A Survey](https://mlsanity.com/paper/2011.00416/) (2020)
- [CausalGAN: Learning Causal Implicit Generative Models with Adversarial Training](https://mlsanity.com/paper/1709.02023/) (2017)
- [Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation](https://mlsanity.com/paper/1911.03842/) (2019)

## Built on by (most cited first, 30 of 45)

- [RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning](https://mlsanity.com/paper/2205.12548/) (2022)
- [Large Language Models with Controllable Working Memory](https://mlsanity.com/paper/2211.05110/) (2022)
- [A Comprehensive Survey on Deep Graph Representation Learning](https://mlsanity.com/paper/2304.05055/) (2023)
- [ChatGPT vs Human-authored Text: Insights into Controllable Text Summarization and Sentence Style Transfer](https://mlsanity.com/paper/2306.07799/) (2023)
- [Mitigating Object Hallucination in Large Vision-Language Models via Image-Grounded Guidance](https://mlsanity.com/paper/2402.08680/) (2024)
- [Neuro-Symbolic Procedural Planning with Commonsense Prompting](https://mlsanity.com/paper/2206.02928/) (2022)
- [Composable Text Controls in Latent Space with ODEs](https://mlsanity.com/paper/2208.00638/) (2022)
- [Causally Testing Gender Bias in LLMs: A Case Study on Occupational Bias](https://mlsanity.com/paper/2212.10678/) (2022)
- [NeuroCounterfactuals: Beyond Minimal-Edit Counterfactuals for Richer Data Augmentation](https://mlsanity.com/paper/2210.12365/) (2022)
- [A Survey on Natural Language Counterfactual Generation](https://mlsanity.com/paper/2407.03993/) (2024)
- [Controllable Data Generation by Deep Learning: A Review](https://mlsanity.com/paper/2207.09542/) (2022)
- [Controllable Text Summarization: Unraveling Challenges, Approaches, and Prospects -- A Survey](https://mlsanity.com/paper/2311.09212/) (2023)
- [Causal Inference with Generative Artificial Intelligence: Application to Texts as Treatments](https://mlsanity.com/paper/2410.00903/) (2024)
- [Cross-Modal Causal Intervention for Medical Report Generation](https://mlsanity.com/paper/2303.09117/) (2023)
- [MacLaSa: Multi-Aspect Controllable Text Generation via Efficient Sampling from Compact Latent Space](https://mlsanity.com/paper/2305.12785/) (2023)
- [Emerging Synergies in Causality and Deep Generative Models: A Survey](https://mlsanity.com/paper/2301.12351/) (2023)
- [An Invariant Learning Characterization of Controlled Text Generation](https://mlsanity.com/paper/2306.00198/) (2023)
- [Deep Backtracking Counterfactuals for Causally Compliant Explanations](https://mlsanity.com/paper/2310.07665/) (2023)
- [Causality-aware Concept Extraction based on Knowledge-guided Prompting](https://mlsanity.com/paper/2305.01876/) (2023)
- [Causal Time Series Generation via Diffusion Models](https://mlsanity.com/paper/2509.20846/) (2025)
- [FAST: Improving Controllability for Text Generation with Feedback Aware Self-Training](https://mlsanity.com/paper/2210.03167/) (2022)
- [Learning Disentangled Semantic Spaces of Explanations via Invertible Neural Networks](https://mlsanity.com/paper/2305.01713/) (2023)
- [Graph-Induced Syntactic-Semantic Spaces in Transformer-Based Variational AutoEncoders](https://mlsanity.com/paper/2311.08579/) (2023)
- [Improving Semantic Control in Discrete Latent Spaces with Transformer Quantized Variational Autoencoders](https://mlsanity.com/paper/2402.00723/) (2024)
- [Preference Learning for AI Alignment: a Causal Perspective](https://mlsanity.com/paper/2506.05967/) (2025)
- [Text Transformations in Contrastive Self-Supervised Learning: A Review](https://mlsanity.com/paper/2203.12000/) (2022)
- [Quasi-symbolic Semantic Geometry over Transformer-based Variational AutoEncoder](https://mlsanity.com/paper/2210.06230/) (2022)
- [Narrating Causal Graphs with Large Language Models](https://mlsanity.com/paper/2403.07118/) (2024)
- [MambaTAD: When State-Space Models Meet Long-Range Temporal Action Detection](https://mlsanity.com/paper/2511.17929/) (2025)
- [Harnessing Knowledge and Reasoning for Human-Like Natural Language Generation: A Brief Review](https://mlsanity.com/paper/2212.03747/) (2022)

## Abstract (arXiv)

Controllable text generation concerns two fundamental tasks of wide applications, namely generating text of given attributes (i.e., attribute-conditional generation), and minimally editing existing text to possess desired attributes (i.e., text attribute transfer). Extensive prior work has largely studied the two problems separately, and developed different conditional models which, however, are prone to producing biased text (e.g., various gender stereotypes). This paper proposes to formulate controllable text generation from a principled causal perspective which models the two tasks with a unified framework. A direct advantage of the causal formulation is the use of rich causality tools to mitigate generation biases and improve control. We treat the two tasks as interventional and counterfactual causal inference based on a structural causal model, respectively. We then apply the framework to the challenging practical setting where confounding factors (that induce spurious correlations) are observable only on a small fraction of data. Experiments show significant superiority of the causal approach over previous conditional models for improved control accuracy and reduced bias.