# Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer

arXiv: https://arxiv.org/abs/1804.06437 · HTML: https://mlsanity.com/paper/1804.06437/

## Key facts

- Cited by 191 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Apr 2018.

## Built on by (most cited first, 30 of 191)

- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts](https://mlsanity.com/paper/2105.03023/) (2021)
- [RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning](https://mlsanity.com/paper/2205.12548/) (2022)
- [Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer](https://mlsanity.com/paper/2110.07139/) (2021)
- [A Survey on Retrieval-Augmented Text Generation](https://mlsanity.com/paper/2202.01110/) (2022)
- [A Recipe For Arbitrary Text Style Transfer with Large Language Models](https://mlsanity.com/paper/2109.03910/) (2021)
- [The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generation](https://mlsanity.com/paper/2109.06835/) (2021)
- [Controlled Text Generation with Natural Language Instructions](https://mlsanity.com/paper/2304.14293/) (2023)
- [Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model](https://mlsanity.com/paper/2310.09520/) (2023)
- [Towards Facilitating Empathic Conversations in Online Mental Health Support: A Reinforcement Learning Approach](https://mlsanity.com/paper/2101.07714/) (2021)
- [Controlled Text Generation as Continuous Optimization with Multiple Constraints](https://mlsanity.com/paper/2108.01850/) (2021)
- [Word Embeddings Are Steers for Language Models](https://mlsanity.com/paper/2305.12798/) (2023)
- [Latent Diffusion Energy-Based Model for Interpretable Text Modeling](https://mlsanity.com/paper/2206.05895/) (2022)
- [Detecting and Grounding Multi-Modal Media Manipulation and Beyond](https://mlsanity.com/paper/2309.14203/) (2023)
- [XFORMAL: A Benchmark for Multilingual Formality Style Transfer](https://mlsanity.com/paper/2104.04108/) (2021)
- [Text Detoxification using Large Pre-trained Neural Models](https://mlsanity.com/paper/2109.08914/) (2021)
- [Prompt-and-Rerank: A Method for Zero-Shot and Few-Shot Arbitrary Textual Style Transfer with Small Language Models](https://mlsanity.com/paper/2205.11503/) (2022)
- [Teach LLMs to Personalize -- An Approach inspired by Writing Education](https://mlsanity.com/paper/2308.07968/) (2023)
- [Less is More: Learning to Refine Dialogue History for Personalized Dialogue Generation](https://mlsanity.com/paper/2204.08128/) (2022)
- [A Causal Lens for Controllable Text Generation](https://mlsanity.com/paper/2201.09119/) (2022)
- [Benchmarking and Improving Generator-Validator Consistency of Language Models](https://mlsanity.com/paper/2310.01846/) (2023)
- [LEWIS: Levenshtein Editing for Unsupervised Text Style Transfer](https://mlsanity.com/paper/2105.08206/) (2021)
- [A Novel Estimator of Mutual Information for Learning to Disentangle Textual Representations](https://mlsanity.com/paper/2105.02685/) (2021)
- [Detoxifying Text with MaRCo: Controllable Revision with Experts and Anti-Experts](https://mlsanity.com/paper/2212.10543/) (2022)
- [StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer](https://mlsanity.com/paper/2104.05196/) (2021)
- [Civil Rephrases Of Toxic Texts With Self-Supervised Transformers](https://mlsanity.com/paper/2102.05456/) (2021)
- [Meet Your Favorite Character: Open-domain Chatbot Mimicking Fictional Characters with only a Few Utterances](https://mlsanity.com/paper/2204.10825/) (2022)
- [Inducing Positive Perspectives with Text Reframing](https://mlsanity.com/paper/2204.02952/) (2022)

## Abstract (arXiv)

We consider the task of text attribute transfer: transforming a sentence to alter a specific attribute (e.g., sentiment) while preserving its attribute-independent content (e.g., changing "screen is just the right size" to "screen is too small"). Our training data includes only sentences labeled with their attribute (e.g., positive or negative), but not pairs of sentences that differ only in their attributes, so we must learn to disentangle attributes from attribute-independent content in an unsupervised way. Previous work using adversarial methods has struggled to produce high-quality outputs. In this paper, we propose simpler methods motivated by the observation that text attributes are often marked by distinctive phrases (e.g., "too small"). Our strongest method extracts content words by deleting phrases associated with the sentence's original attribute value, retrieves new phrases associated with the target attribute, and uses a neural model to fluently combine these into a final output. On human evaluation, our best method generates grammatical and appropriate responses on 22% more inputs than the best previous system, averaged over three attribute transfer datasets: altering sentiment of reviews on Yelp, altering sentiment of reviews on Amazon, and altering image captions to be more romantic or humorous.