# Generic resources are what you need: Style transfer tasks without task-specific parallel training data arXiv: https://arxiv.org/abs/2109.04543 ยท HTML: https://mlsanity.com/paper/2109.04543/ ## Key facts - Cited by 10 papers in the map (data through 14 Oct 2026). - Builds on 26 of its 40 references that are in the map. - First posted on arXiv 9 Sep 2021. ## Builds on (most cited first, 26 of 26) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019) - [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014) - [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013) - [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020) - [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014) - [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020) - [Style Transfer from Non-Parallel Text by Cross-Alignment](https://mlsanity.com/paper/1705.09655/) (2017) - [Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer](https://mlsanity.com/paper/1803.06535/) (2018) - [Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer](https://mlsanity.com/paper/1804.06437/) (2018) - [Style Transfer in Text: Exploration and Evaluation](https://mlsanity.com/paper/1711.06861/) (2017) - [Disentangled Representation Learning for Non-Parallel Text Style Transfer](https://mlsanity.com/paper/1808.04339/) (2018) - [Style Transfer Through Back-Translation](https://mlsanity.com/paper/1804.09000/) (2018) - [Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation](https://mlsanity.com/paper/1905.05621/) (2019) - [A Dual Reinforcement Learning Framework for Unsupervised Text Style Transfer](https://mlsanity.com/paper/1905.10060/) (2019) - [Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach](https://mlsanity.com/paper/1805.05181/) (2018) - [Regularizing Neural Machine Translation by Target-bidirectional Agreement](https://mlsanity.com/paper/1808.04064/) (2018) - [Unsupervised Paraphrasing by Simulated Annealing](https://mlsanity.com/paper/1909.03588/) (2019) - [Unsupervised Text Generation by Learning from Search](https://mlsanity.com/paper/2007.08557/) (2020) - [Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer](https://mlsanity.com/paper/2105.06947/) (2021) - [Stable Style Transformer: Delete and Generate Approach with Encoder-Decoder for Text Style Transfer](https://mlsanity.com/paper/2005.12086/) (2020) - [A Hierarchical Reinforced Sequence Operation Method for Unsupervised Text Style Transfer](https://mlsanity.com/paper/1906.01833/) (2019) - [Semi-supervised Formality Style Transfer using Language Model Discriminator and Mutual Information Maximization](https://mlsanity.com/paper/2010.05090/) (2020) - [Exploring Contextual Word-level Style Relevance for Unsupervised Style Transfer](https://mlsanity.com/paper/2005.02049/) (2020) - [Reinforced Rewards Framework for Text Style Transfer](https://mlsanity.com/paper/2005.05256/) (2020) ## Built on by (most cited first, 10 of 10) - [Meet Your Favorite Character: Open-domain Chatbot Mimicking Fictional Characters with only a Few Utterances](https://mlsanity.com/paper/2204.10825/) (2022) - [A Call for Standardization and Validation of Text Style Transfer Evaluation](https://mlsanity.com/paper/2306.00539/) (2023) - [Human Judgement as a Compass to Navigate Automatic Metrics for Formality Transfer](https://mlsanity.com/paper/2204.07549/) (2022) - [Multidimensional Evaluation for Text Style Transfer Using ChatGPT](https://mlsanity.com/paper/2304.13462/) (2023) - [Multilingual Pre-training with Language and Task Adaptation for Multilingual Text Style Transfer](https://mlsanity.com/paper/2203.08552/) (2022) - [Multi-Figurative Language Generation](https://mlsanity.com/paper/2209.01835/) (2022) - [Responsibility Perspective Transfer for Italian Femicide News](https://mlsanity.com/paper/2306.00437/) (2023) - [Multi-perspective Alignment for Increasing Naturalness in Neural Machine Translation](https://mlsanity.com/paper/2412.08473/) (2024) - [MSSRNet: Manipulating Sequential Style Representation for Unsupervised Text Style Transfer](https://mlsanity.com/paper/2306.07994/) (2023) - [StyleBART: Decorate Pretrained Model with Style Adapters for Unsupervised Stylistic Headline Generation](https://mlsanity.com/paper/2310.17743/) (2023) ## Abstract (arXiv) Style transfer aims to rewrite a source text in a different target style while preserving its content. We propose a novel approach to this task that leverages generic resources, and without using any task-specific parallel (source-target) data outperforms existing unsupervised approaches on the two most popular style transfer tasks: formality transfer and polarity swap. In practice, we adopt a multi-step procedure which builds on a generic pre-trained sequence-to-sequence model (BART). First, we strengthen the model's ability to rewrite by further pre-training BART on both an existing collection of generic paraphrases, as well as on synthetic pairs created using a general-purpose lexical resource. Second, through an iterative back-translation approach, we train two models, each in a transfer direction, so that they can provide each other with synthetically generated pairs, dynamically in the training process. Lastly, we let our best reresulting model generate static synthetic pairs to be used in a supervised training regime. Besides methodology and state-of-the-art results, a core contribution of this work is a reflection on the nature of the two tasks we address, and how their differences are highlighted by their response to our approach.