# On Learning Text Style Transfer with Direct Rewards

arXiv: https://arxiv.org/abs/2010.12771 · HTML: https://mlsanity.com/paper/2010.12771/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Oct 2020.

## Built on by (most cited first, 24 of 24)

- [RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning](https://mlsanity.com/paper/2205.12548/) (2022)
- [StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer](https://mlsanity.com/paper/2104.05196/) (2021)
- [Composable Text Controls in Latent Space with ODEs](https://mlsanity.com/paper/2208.00638/) (2022)
- [Evaluating the Evaluation Metrics for Style Transfer: A Case Study in Multilingual Formality Transfer](https://mlsanity.com/paper/2110.10668/) (2021)
- [Enhancing Presentation Slide Generation by LLMs with a Multi-Staged End-to-End Approach](https://mlsanity.com/paper/2406.06556/) (2024)
- [Controlling Personality Style in Dialogue with Zero-Shot Prompt-Based Learning](https://mlsanity.com/paper/2302.03848/) (2023)
- [A Call for Standardization and Validation of Text Style Transfer Evaluation](https://mlsanity.com/paper/2306.00539/) (2023)
- [Authorship Style Transfer with Policy Optimization](https://mlsanity.com/paper/2403.08043/) (2024)
- [Presentations are not always linear! GNN meets LLM for Document-to-Presentation Transformation with Attribution](https://mlsanity.com/paper/2405.13095/) (2024)
- [Improving Large-scale Paraphrase Acquisition and Generation](https://mlsanity.com/paper/2210.03235/) (2022)
- [LLM-based Rewriting of Inappropriate Argumentation using Reinforcement Learning from Machine Feedback](https://mlsanity.com/paper/2406.03363/) (2024)
- [Attribution, Citation, and Quotation: A Survey of Evidence-based Text Generation with Large Language Models](https://mlsanity.com/paper/2508.15396/) (2025)
- [T-STAR: Truthful Style Transfer using AMR Graph as Intermediate Representation](https://mlsanity.com/paper/2212.01667/) (2022)
- [Style Transfer with Multi-iteration Preference Optimization](https://mlsanity.com/paper/2406.11581/) (2024)
- [Syntax Matters! Syntax-Controlled in Text Style Transfer](https://mlsanity.com/paper/2108.05869/) (2021)
- [Entailment Relation Aware Paraphrase Generation](https://mlsanity.com/paper/2203.10483/) (2022)
- [Stylistic Retrieval-based Dialogue System with Unparallel Training Data](https://mlsanity.com/paper/2109.05477/) (2021)
- [Style Transfer as Data Augmentation: A Case Study on Named Entity Recognition](https://mlsanity.com/paper/2210.07916/) (2022)
- [Multi-Agent Synergy-Driven Iterative Visual Narrative Synthesis](https://mlsanity.com/paper/2507.13285/) (2025)
- [Learning from Bootstrapping and Stepwise Reinforcement Reward: A Semi-Supervised Framework for Text Style Transfer](https://mlsanity.com/paper/2205.09324/) (2022)
- [On Text Style Transfer via Style Masked Language Models](https://mlsanity.com/paper/2210.06394/) (2022)
- [Learning a Formality-Aware Japanese Sentence Representation](https://mlsanity.com/paper/2301.07209/) (2023)
- [MSSRNet: Manipulating Sequential Style Representation for Unsupervised Text Style Transfer](https://mlsanity.com/paper/2306.07994/) (2023)
- [Reward Dropout Improves Control: Bi-objective Perspective on Reinforced LM](https://mlsanity.com/paper/2310.04483/) (2023)

## Abstract (arXiv)

In most cases, the lack of parallel corpora makes it impossible to directly train supervised models for the text style transfer task. In this paper, we explore training algorithms that instead optimize reward functions that explicitly consider different aspects of the style-transferred outputs. In particular, we leverage semantic similarity metrics originally used for fine-tuning neural machine translation models to explicitly assess the preservation of content between system outputs and input texts. We also investigate the potential weaknesses of the existing automatic metrics and propose efficient strategies of using these metrics for training. The experimental results show that our model provides significant gains in both automatic and human evaluation over strong baselines, indicating the effectiveness of our proposed methods and training strategies.