# Prompt-and-Rerank: A Method for Zero-Shot and Few-Shot Arbitrary Textual Style Transfer with Small Language Models

arXiv: https://arxiv.org/abs/2205.11503 · HTML: https://mlsanity.com/paper/2205.11503/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 47 references that are in the map.
- First posted on arXiv 23 May 2022.

## Builds on (most cited first, 30 of 33)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [BARTScore: Evaluating Generated Text as Text Generation](https://mlsanity.com/paper/2106.11520/) (2021)
- [Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering](https://mlsanity.com/paper/1602.01585/) (2016)
- [MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance](https://mlsanity.com/paper/1909.02622/) (2019)
- [Toward Controlled Generation of Text](https://mlsanity.com/paper/1703.00955/) (2017)
- [Style Transfer from Non-Parallel Text by Cross-Alignment](https://mlsanity.com/paper/1705.09655/) (2017)
- [Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer](https://mlsanity.com/paper/1803.06535/) (2018)
- [Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer](https://mlsanity.com/paper/1804.06437/) (2018)
- [Reformulating Unsupervised Style Transfer as Paraphrase Generation](https://mlsanity.com/paper/2010.05700/) (2020)
- [Deep Learning for Text Style Transfer: A Survey](https://mlsanity.com/paper/2011.00416/) (2020)
- [Style Transfer in Text: Exploration and Evaluation](https://mlsanity.com/paper/1711.06861/) (2017)
- [Style Transfer Through Back-Translation](https://mlsanity.com/paper/1804.09000/) (2018)
- [A Recipe For Arbitrary Text Style Transfer with Large Language Models](https://mlsanity.com/paper/2109.03910/) (2021)
- [JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correction](https://mlsanity.com/paper/1702.04066/) (2017)
- [Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation](https://mlsanity.com/paper/1905.05621/) (2019)
- [Transforming Delete, Retrieve, Generate Approach for Controlled Text Style Transfer](https://mlsanity.com/paper/1908.09368/) (2019)
- [Politeness Transfer: A Tag and Generate Approach](https://mlsanity.com/paper/2004.14257/) (2020)
- [A Dual Reinforcement Learning Framework for Unsupervised Text Style Transfer](https://mlsanity.com/paper/1905.10060/) (2019)
- [Evaluating Style Transfer for Text](https://mlsanity.com/paper/1904.02295/) (2019)
- [Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach](https://mlsanity.com/paper/1805.05181/) (2018)
- ["Mask and Infill" : Applying Masked Language Model to Sentiment Transfer](https://mlsanity.com/paper/1908.08039/) (2019)

## Built on by (most cited first, 30 of 49)

- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models](https://mlsanity.com/paper/2302.13439/) (2023)
- [Follow the Wisdom of the Crowd: Effective Text Generation via Minimum Bayes Risk Decoding](https://mlsanity.com/paper/2211.07634/) (2022)
- [Tree-Planner: Efficient Close-loop Task Planning with Large Language Models](https://mlsanity.com/paper/2310.08582/) (2023)
- [On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex](https://mlsanity.com/paper/2301.12868/) (2023)
- [Multi-property Steering of Large Language Models with Dynamic Activation Composition](https://mlsanity.com/paper/2406.17563/) (2024)
- [Comparing Biases and the Impact of Multilingual Training across Multiple Languages](https://mlsanity.com/paper/2305.11242/) (2023)
- [StyleDistance: Stronger Content-Independent Style Embeddings with Synthetic Parallel Examples](https://mlsanity.com/paper/2410.12757/) (2024)
- [Deductive Closure Training of Language Models for Coherence, Accuracy, and Updatability](https://mlsanity.com/paper/2401.08574/) (2024)
- [Small Language Models Improve Giants by Rewriting Their Outputs](https://mlsanity.com/paper/2305.13514/) (2023)
- [TinyStyler: Efficient Few-Shot Text Style Transfer with Authorship Embeddings](https://mlsanity.com/paper/2406.15586/) (2024)
- [Prompt-Based Editing for Text Style Transfer](https://mlsanity.com/paper/2301.11997/) (2023)
- [Controlling Personality Style in Dialogue with Zero-Shot Prompt-Based Learning](https://mlsanity.com/paper/2302.03848/) (2023)
- [STEER: Unified Style Transfer with Expert Reinforcement](https://mlsanity.com/paper/2311.07167/) (2023)
- [Characterizing Attribution and Fluency Tradeoffs for Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2302.05578/) (2023)
- [Are Large Language Models Actually Good at Text Style Transfer?](https://mlsanity.com/paper/2406.05885/) (2024)
- [Distilling Text Style Transfer With Self-Explanation From LLMs](https://mlsanity.com/paper/2403.01106/) (2024)
- [Authorship Style Transfer with Policy Optimization](https://mlsanity.com/paper/2403.08043/) (2024)
- [RAMP: Retrieval and Attribute-Marking Enhanced Prompting for Attribute-Controlled Translation](https://mlsanity.com/paper/2305.17131/) (2023)
- [Evaluating Prompting Strategies for Grammatical Error Correction Based on Language Proficiency](https://mlsanity.com/paper/2402.15930/) (2024)
- [Applying Large Language Models for Causal Structure Learning in Non Small Cell Lung Cancer](https://mlsanity.com/paper/2311.07191/) (2023)
- [Improving Black-box Robustness with In-Context Rewriting](https://mlsanity.com/paper/2402.08225/) (2024)
- [Towards Robust NLG Bias Evaluation with Syntactically-diverse Prompts](https://mlsanity.com/paper/2212.01700/) (2022)
- [Unsupervised Text Style Transfer via LLMs and Attention Masking with Multi-way Interactions](https://mlsanity.com/paper/2402.13647/) (2024)
- [ICLEF: In-Context Learning with Expert Feedback for Explainable Style Transfer](https://mlsanity.com/paper/2309.08583/) (2023)
- [LMStyle Benchmark: Evaluating Text Style Transfer for Chatbots](https://mlsanity.com/paper/2403.08943/) (2024)
- [Text Style Transfer: An Introductory Overview](https://mlsanity.com/paper/2407.14822/) (2024)
- [Steering Large Language Models with Register Analysis for Arbitrary Style Transfer](https://mlsanity.com/paper/2505.00679/) (2025)
- [StyleFlow: Disentangle Latent Representations via Normalizing Flow for Unsupervised Text Style Transfer](https://mlsanity.com/paper/2212.09670/) (2022)
- [Conversation Style Transfer using Few-Shot Learning](https://mlsanity.com/paper/2302.08362/) (2023)

## Abstract (arXiv)

We propose a method for arbitrary textual style transfer (TST)--the task of transforming a text into any given style--utilizing general-purpose pre-trained language models. Our method, Prompt-and-Rerank, is based on a mathematical formulation of the TST task, decomposing it into three constituent components: textual similarity, target style strength, and fluency. Specifically, our method first uses zero-shot or few-shot prompting to obtain a set of candidate generations in the target style, and then re-ranks these candidates according to a combination of the three components above. Empirically, our method enables small pre-trained language models to perform on par with state-of-the-art large-scale models while consuming two orders of magnitude less compute and memory. Finally, we conduct a systematic investigation of the effect of model size and prompt design (e.g., prompt paraphrasing and delimiter-pair choice) on style transfer quality across seven diverse textual style transfer datasets.