# LEWIS: Levenshtein Editing for Unsupervised Text Style Transfer

arXiv: https://arxiv.org/abs/2105.08206 · HTML: https://mlsanity.com/paper/2105.08206/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 37 references that are in the map.
- First posted on arXiv 18 May 2021.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Mixed Precision Training](https://mlsanity.com/paper/1710.03740/) (2017)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering](https://mlsanity.com/paper/1602.01585/) (2016)
- [Style Transfer from Non-Parallel Text by Cross-Alignment](https://mlsanity.com/paper/1705.09655/) (2017)
- [Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer](https://mlsanity.com/paper/1804.06437/) (2018)
- [Reformulating Unsupervised Style Transfer as Paraphrase Generation](https://mlsanity.com/paper/2010.05700/) (2020)
- [Revisiting Self-Training for Neural Sequence Generation](https://mlsanity.com/paper/1909.13788/) (2019)
- [Style Transfer in Text: Exploration and Evaluation](https://mlsanity.com/paper/1711.06861/) (2017)
- [Disentangled Representation Learning for Non-Parallel Text Style Transfer](https://mlsanity.com/paper/1808.04339/) (2018)
- [Style Transfer Through Back-Translation](https://mlsanity.com/paper/1804.09000/) (2018)
- [Coarse-to-Fine Decoding for Neural Semantic Parsing](https://mlsanity.com/paper/1805.04793/) (2018)
- [Encode, Tag, Realize: High-Precision Text Editing](https://mlsanity.com/paper/1909.01187/) (2019)
- [Ultra-Fine Entity Typing](https://mlsanity.com/paper/1807.04905/) (2018)
- [Politeness Transfer: A Tag and Generate Approach](https://mlsanity.com/paper/2004.14257/) (2020)
- [A Probabilistic Formulation of Unsupervised Text Style Transfer](https://mlsanity.com/paper/2002.03912/) (2020)
- ["Mask and Infill" : Applying Masked Language Model to Sentiment Transfer](https://mlsanity.com/paper/1908.08039/) (2019)

## Built on by (most cited first, 30 of 40)

- [Tailor: Generating and Perturbing Text with Semantic Controls](https://mlsanity.com/paper/2107.07150/) (2021)
- [Prompt-and-Rerank: A Method for Zero-Shot and Few-Shot Arbitrary Textual Style Transfer with Small Language Models](https://mlsanity.com/paper/2205.11503/) (2022)
- [DiffusER: Discrete Diffusion via Edit-based Reconstruction](https://mlsanity.com/paper/2210.16886/) (2022)
- [On Improving Summarization Factual Consistency from Natural Language Feedback](https://mlsanity.com/paper/2212.09968/) (2022)
- [EdiT5: Semi-Autoregressive Text-Editing with T5 Warm-Start](https://mlsanity.com/paper/2205.12209/) (2022)
- [Learning to Model Editing Processes](https://mlsanity.com/paper/2205.12374/) (2022)
- [Don't lose the message while paraphrasing: A study on content preserving style transfer](https://mlsanity.com/paper/2308.09055/) (2023)
- [Text Generation with Text-Editing Models](https://mlsanity.com/paper/2206.07043/) (2022)
- [A Survey on Natural Language Counterfactual Generation](https://mlsanity.com/paper/2407.03993/) (2024)
- [Improving Iterative Text Revision by Learning Where to Edit from Other Revision Tasks](https://mlsanity.com/paper/2212.01350/) (2022)
- [Text Revision by On-the-Fly Representation Optimization](https://mlsanity.com/paper/2204.07359/) (2022)
- [Proactively Reducing the Hate Intensity of Online Posts via Hate Speech Normalization](https://mlsanity.com/paper/2206.04007/) (2022)
- [Prompt-Based Editing for Text Style Transfer](https://mlsanity.com/paper/2301.11997/) (2023)
- [Human Judgement as a Compass to Navigate Automatic Metrics for Formality Transfer](https://mlsanity.com/paper/2204.07549/) (2022)
- [Distilling Text Style Transfer With Self-Explanation From LLMs](https://mlsanity.com/paper/2403.01106/) (2024)
- [Authorship Style Transfer with Policy Optimization](https://mlsanity.com/paper/2403.08043/) (2024)
- [DetoxLLM: A Framework for Detoxification with Explanations](https://mlsanity.com/paper/2402.15951/) (2024)
- [Non-autoregressive Text Editing with Copy-aware Latent Alignments](https://mlsanity.com/paper/2310.07821/) (2023)
- [Towards Robust and Semantically Organised Latent Representations for Unsupervised Text Style Transfer](https://mlsanity.com/paper/2205.02309/) (2022)
- [Unsupervised Text Style Transfer via LLMs and Attention Masking with Multi-way Interactions](https://mlsanity.com/paper/2402.13647/) (2024)
- [LLM-based Rewriting of Inappropriate Argumentation using Reinforcement Learning from Machine Feedback](https://mlsanity.com/paper/2406.03363/) (2024)
- [GRS: Combining Generation and Revision in Unsupervised Sentence Simplification](https://mlsanity.com/paper/2203.09742/) (2022)
- [Don't Take This Out of Context! On the Need for Contextual Models and Evaluations for Stylistic Rewriting](https://mlsanity.com/paper/2305.14755/) (2023)
- [From Theories on Styles to their Transfer in Text: Bridging the Gap with a Hierarchical Survey](https://mlsanity.com/paper/2110.15871/) (2021)
- [Human-Guided Fair Classification for Natural Language Processing](https://mlsanity.com/paper/2212.10154/) (2022)
- [LMStyle Benchmark: Evaluating Text Style Transfer for Chatbots](https://mlsanity.com/paper/2403.08943/) (2024)
- [StyleFlow: Disentangle Latent Representations via Normalizing Flow for Unsupervised Text Style Transfer](https://mlsanity.com/paper/2212.09670/) (2022)
- [Personalized Abstractive Summarization by Tri-agent Generation Pipeline](https://mlsanity.com/paper/2305.02483/) (2023)
- [Style Transfer with Multi-iteration Preference Optimization](https://mlsanity.com/paper/2406.11581/) (2024)
- [Replacing Language Model for Style Transfer](https://mlsanity.com/paper/2211.07343/) (2022)

## Abstract (arXiv)

Many types of text style transfer can be achieved with only small, precise edits (e.g. sentiment transfer from I had a terrible time... to I had a great time...). We propose a coarse-to-fine editor for style transfer that transforms text using Levenshtein edit operations (e.g. insert, replace, delete). Unlike prior single-span edit methods, our method concurrently edits multiple spans in the source text. To train without parallel style text pairs (e.g. pairs of +/- sentiment statements), we propose an unsupervised data synthesis procedure. We first convert text to style-agnostic templates using style classifier attention (e.g. I had a SLOT time...), then fill in slots in these templates using fine-tuned pretrained language models. Our method outperforms existing generation and editing style transfer methods on sentiment (Yelp, Amazon) and politeness (Polite) transfer. In particular, multi-span editing achieves higher performance and more diverse output than single-span editing. Moreover, compared to previous methods on unsupervised data synthesis, our method results in higher quality parallel style pairs and improves model performance.