# A Dual Reinforcement Learning Framework for Unsupervised Text Style Transfer

arXiv: https://arxiv.org/abs/1905.10060 · HTML: https://mlsanity.com/paper/1905.10060/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 May 2019.

## Built on by (most cited first, 30 of 65)

- [RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning](https://mlsanity.com/paper/2205.12548/) (2022)
- [Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer](https://mlsanity.com/paper/2110.07139/) (2021)
- [A Recipe For Arbitrary Text Style Transfer with Large Language Models](https://mlsanity.com/paper/2109.03910/) (2021)
- [Towards Facilitating Empathic Conversations in Online Mental Health Support: A Reinforcement Learning Approach](https://mlsanity.com/paper/2101.07714/) (2021)
- [XFORMAL: A Benchmark for Multilingual Formality Style Transfer](https://mlsanity.com/paper/2104.04108/) (2021)
- [Text Detoxification using Large Pre-trained Neural Models](https://mlsanity.com/paper/2109.08914/) (2021)
- [Counterfactual Cycle-Consistent Learning for Instruction Following and Generation in Vision-Language Navigation](https://mlsanity.com/paper/2203.16586/) (2022)
- [Prompt-and-Rerank: A Method for Zero-Shot and Few-Shot Arbitrary Textual Style Transfer with Small Language Models](https://mlsanity.com/paper/2205.11503/) (2022)
- [A Causal Lens for Controllable Text Generation](https://mlsanity.com/paper/2201.09119/) (2022)
- [You Only Prompt Once: On the Capabilities of Prompt Learning on Large Language Models to Tackle Toxic Content](https://mlsanity.com/paper/2308.05596/) (2023)
- [Civil Rephrases Of Toxic Texts With Self-Supervised Transformers](https://mlsanity.com/paper/2102.05456/) (2021)
- [Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer](https://mlsanity.com/paper/2105.06947/) (2021)
- [Evaluating the Evaluation Metrics for Style Transfer: A Case Study in Multilingual Formality Transfer](https://mlsanity.com/paper/2110.10668/) (2021)
- [Reinforcement Learning for Emotional Text-to-Speech Synthesis with Improved Emotion Discriminability](https://mlsanity.com/paper/2104.01408/) (2021)
- [Text Revision by On-the-Fly Representation Optimization](https://mlsanity.com/paper/2204.07359/) (2022)
- [Prompt-Based Editing for Text Style Transfer](https://mlsanity.com/paper/2301.11997/) (2023)
- [A Call for Standardization and Validation of Text Style Transfer Evaluation](https://mlsanity.com/paper/2306.00539/) (2023)
- [STEER: Unified Style Transfer with Expert Reinforcement](https://mlsanity.com/paper/2311.07167/) (2023)
- [MOVER: Mask, Over-generate and Rank for Hyperbole Generation](https://mlsanity.com/paper/2109.07726/) (2021)
- [A Survey of Natural Language Generation](https://mlsanity.com/paper/2112.11739/) (2021)
- [Style Pooling: Automatic Text Style Obfuscation for Improved Classification Fairness](https://mlsanity.com/paper/2109.04624/) (2021)
- [Transductive Learning for Unsupervised Text Style Transfer](https://mlsanity.com/paper/2109.07812/) (2021)
- [Semi-Supervised Formality Style Transfer with Consistency Training](https://mlsanity.com/paper/2203.13620/) (2022)
- [Human Judgement as a Compass to Navigate Automatic Metrics for Formality Transfer](https://mlsanity.com/paper/2204.07549/) (2022)
- [Multidimensional Evaluation for Text Style Transfer Using ChatGPT](https://mlsanity.com/paper/2304.13462/) (2023)
- [Distilling Text Style Transfer With Self-Explanation From LLMs](https://mlsanity.com/paper/2403.01106/) (2024)
- [Non-Parallel Text Style Transfer with Self-Parallel Supervision](https://mlsanity.com/paper/2204.08123/) (2022)
- [Generic resources are what you need: Style transfer tasks without task-specific parallel training data](https://mlsanity.com/paper/2109.04543/) (2021)
- [Transferable Persona-Grounded Dialogues via Grounded Minimal Edits](https://mlsanity.com/paper/2109.07713/) (2021)
- [Multilingual Pre-training with Language and Task Adaptation for Multilingual Text Style Transfer](https://mlsanity.com/paper/2203.08552/) (2022)

## Abstract (arXiv)

Unsupervised text style transfer aims to transfer the underlying style of text but keep its main content unchanged without parallel data. Most existing methods typically follow two steps: first separating the content from the original style, and then fusing the content with the desired style. However, the separation in the first step is challenging because the content and style interact in subtle ways in natural language. Therefore, in this paper, we propose a dual reinforcement learning framework to directly transfer the style of the text via a one-step mapping model, without any separation of content and style. Specifically, we consider the learning of the source-to-target and target-to-source mappings as a dual task, and two rewards are designed based on such a dual structure to reflect the style accuracy and content preservation, respectively. In this way, the two one-step mapping models can be trained via reinforcement learning, without any use of parallel data. Automatic evaluations show that our model outperforms the state-of-the-art systems by a large margin, especially with more than 8 BLEU points improvement averaged on two benchmark datasets. Human evaluations also validate the effectiveness of our model in terms of style accuracy, content preservation and fluency. Our code and data, including outputs of all baselines and our model are available at https://github.com/luofuli/DualLanST.