# FAST: Improving Controllability for Text Generation with Feedback Aware Self-Training

arXiv: https://arxiv.org/abs/2210.03167 · HTML: https://mlsanity.com/paper/2210.03167/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 61 references that are in the map.
- First posted on arXiv 6 Oct 2022.

## Builds on (most cited first, 30 of 36)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [Deduplicating Training Data Makes Language Models Better](https://mlsanity.com/paper/2107.06499/) (2021)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [Learning the Difference that Makes a Difference with Counterfactually-Augmented Data](https://mlsanity.com/paper/1909.12434/) (2019)
- [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021)
- [GeDi: Generative Discriminator Guided Sequence Generation](https://mlsanity.com/paper/2009.06367/) (2020)
- [FUDGE: Controlled Text Generation With Future Discriminators](https://mlsanity.com/paper/2104.05218/) (2021)
- [Recommendations as Treatments: Debiasing Learning and Evaluation](https://mlsanity.com/paper/1602.05352/) (2016)
- [Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models](https://mlsanity.com/paper/2101.00288/) (2021)
- [Gender Bias in Neural Natural Language Processing](https://mlsanity.com/paper/1807.11714/) (2018)
- [A Survey on Causal Inference](https://mlsanity.com/paper/2002.02770/) (2020)
- [Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology](https://mlsanity.com/paper/1906.04571/) (2019)
- [Counterfactual Fairness in Text Classification through Robustness](https://mlsanity.com/paper/1809.10610/) (2018)
- [Controlling Linguistic Style Aspects in Neural Language Generation](https://mlsanity.com/paper/1707.02633/) (2017)
- [What makes a good conversation? How controllable attributes affect human judgments](https://mlsanity.com/paper/1902.08654/) (2019)
- [An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models](https://mlsanity.com/paper/2007.06778/) (2020)
- [Controllable Abstractive Summarization](https://mlsanity.com/paper/1711.05217/) (2017)
- [Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Tests](https://mlsanity.com/paper/2106.00545/) (2021)
- [Controlling Output Length in Neural Encoder-Decoders](https://mlsanity.com/paper/1609.09552/) (2016)
- [Controllable Natural Language Generation with Contrastive Prefixes](https://mlsanity.com/paper/2202.13257/) (2022)
- [Improved Zero-shot Neural Machine Translation via Ignoring Spurious Correlations](https://mlsanity.com/paper/1906.01181/) (2019)
- [A Causal Lens for Controllable Text Generation](https://mlsanity.com/paper/2201.09119/) (2022)
- [MReD: A Meta-Review Dataset for Structure-Controllable Text Generation](https://mlsanity.com/paper/2110.07474/) (2021)
- [TextSETTR: Few-Shot Text Style Extraction and Tunable Targeted Restyling](https://mlsanity.com/paper/2010.03802/) (2020)

## Built on by (most cited first, 7 of 7)

- [Natural Language Generation for Advertising: A Survey](https://mlsanity.com/paper/2306.12719/) (2023)
- [ZeroGen: Zero-shot Multimodal Controllable Text Generation with Multiple Oracles](https://mlsanity.com/paper/2306.16649/) (2023)
- [Fine-grained length controllable video captioning with ordinal embeddings](https://mlsanity.com/paper/2408.15447/) (2024)
- [Precise Attribute Intensity Control in Large Language Models via Targeted Representation Editing](https://mlsanity.com/paper/2510.12121/) (2025)
- [Aligning the Query Space: Greedy Information Projection for Language Model Data Selection](https://mlsanity.com/paper/2603.13790/) (2026)
- [DUDE: Diffusion-Based Unsupervised Cross-Domain Image Retrieval](https://mlsanity.com/paper/2509.04193/) (2025)
- [CAPO: Constraint-Aware Prompt Optimization for LLM Agents](https://mlsanity.com/paper/2608.16068/) (2026)

## Abstract (arXiv)

Controllable text generation systems often leverage control codes to direct various properties of the output like style and length. Inspired by recent work on causal inference for NLP, this paper reveals a previously overlooked flaw in these control code-based conditional text generation algorithms. Spurious correlations in the training data can lead models to incorrectly rely on parts of the input other than the control code for attribute selection, significantly undermining downstream generation quality and controllability. We demonstrate the severity of this issue with a series of case studies and then propose two simple techniques to reduce these correlations in training sets. The first technique is based on resampling the data according to an example's propensity towards each linguistic attribute (IPS). The second produces multiple counterfactual versions of each example and then uses an additional feedback mechanism to remove noisy examples (feedback aware self-training, FAST). We evaluate on 3 tasks -- news headline, meta review, and search ads generation -- and demonstrate that FAST can significantly improve the controllability and language quality of generated outputs when compared to state-of-the-art controllable text generation approaches.