# A Hierarchical Reinforced Sequence Operation Method for Unsupervised Text Style Transfer

arXiv: https://arxiv.org/abs/1906.01833 · HTML: https://mlsanity.com/paper/1906.01833/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2019.

## Built on by (most cited first, 20 of 20)

- [XFORMAL: A Benchmark for Multilingual Formality Style Transfer](https://mlsanity.com/paper/2104.04108/) (2021)
- [Civil Rephrases Of Toxic Texts With Self-Supervised Transformers](https://mlsanity.com/paper/2102.05456/) (2021)
- [Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer](https://mlsanity.com/paper/2105.06947/) (2021)
- [A Call for Standardization and Validation of Text Style Transfer Evaluation](https://mlsanity.com/paper/2306.00539/) (2023)
- [Transductive Learning for Unsupervised Text Style Transfer](https://mlsanity.com/paper/2109.07812/) (2021)
- [Multidimensional Evaluation for Text Style Transfer Using ChatGPT](https://mlsanity.com/paper/2304.13462/) (2023)
- [Generic resources are what you need: Style transfer tasks without task-specific parallel training data](https://mlsanity.com/paper/2109.04543/) (2021)
- [Political Depolarization of News Articles Using Attribute-aware Word Embeddings](https://mlsanity.com/paper/2101.01391/) (2021)
- [NAST: A Non-Autoregressive Generator with Word Alignment for Unsupervised Text Style Transfer](https://mlsanity.com/paper/2106.02210/) (2021)
- [Efficient Reinforcement Learning for Unsupervised Controlled Text Generation](https://mlsanity.com/paper/2204.07696/) (2022)
- [LLM-based Rewriting of Inappropriate Argumentation using Reinforcement Learning from Machine Feedback](https://mlsanity.com/paper/2406.03363/) (2024)
- [From Theories on Styles to their Transfer in Text: Bridging the Gap with a Hierarchical Survey](https://mlsanity.com/paper/2110.15871/) (2021)
- [VQA-GEN: A Visual Question Answering Benchmark for Domain Generalization](https://mlsanity.com/paper/2311.00807/) (2023)
- [Stylistic Retrieval-based Dialogue System with Unparallel Training Data](https://mlsanity.com/paper/2109.05477/) (2021)
- [SE-DAE: Style-Enhanced Denoising Auto-Encoder for Unsupervised Text Style Transfer](https://mlsanity.com/paper/2104.12977/) (2021)
- [Empirical Evaluation of Supervision Signals for Style Transfer Models](https://mlsanity.com/paper/2101.06172/) (2021)
- [Learning from Bootstrapping and Stepwise Reinforcement Reward: A Semi-Supervised Framework for Text Style Transfer](https://mlsanity.com/paper/2205.09324/) (2022)
- [On Text Style Transfer via Style Masked Language Models](https://mlsanity.com/paper/2210.06394/) (2022)
- [SimpleStyle: An Adaptable Style Transfer Approach](https://mlsanity.com/paper/2212.10498/) (2022)
- [Teaching LLMs Human-Like Editing of Inappropriate Argumentation via Reinforcement Learning](https://mlsanity.com/paper/2604.12770/) (2026)

## Abstract (arXiv)

Unsupervised text style transfer aims to alter text styles while preserving the content, without aligned data for supervision. Existing seq2seq methods face three challenges: 1) the transfer is weakly interpretable, 2) generated outputs struggle in content preservation, and 3) the trade-off between content and style is intractable. To address these challenges, we propose a hierarchical reinforced sequence operation method, named Point-Then-Operate (PTO), which consists of a high-level agent that proposes operation positions and a low-level agent that alters the sentence. We provide comprehensive training objectives to control the fluency, style, and content of the outputs and a mask-based inference algorithm that allows for multi-step revision based on the single-step trained agents. Experimental results on two text style transfer datasets show that our method significantly outperforms recent methods and effectively addresses the aforementioned challenges.