# Non-Parallel Text Style Transfer with Self-Parallel Supervision

arXiv: https://arxiv.org/abs/2204.08123 · HTML: https://mlsanity.com/paper/2204.08123/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 71 references that are in the map.
- First posted on arXiv 18 Apr 2022.

## Builds on (most cited first, 30 of 51)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [SPICE: Semantic Propositional Image Caption Evaluation](https://mlsanity.com/paper/1607.08822/) (2016)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [Latent Retrieval for Weakly Supervised Open Domain Question Answering](https://mlsanity.com/paper/1906.00300/) (2019)
- [Sequence Level Training with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06732/) (2015)
- [Self-critical Sequence Training for Image Captioning](https://mlsanity.com/paper/1612.00563/) (2016)
- [Style Transfer from Non-Parallel Text by Cross-Alignment](https://mlsanity.com/paper/1705.09655/) (2017)
- [Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer](https://mlsanity.com/paper/1803.06535/) (2018)
- [Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer](https://mlsanity.com/paper/1804.06437/) (2018)
- [Reformulating Unsupervised Style Transfer as Paraphrase Generation](https://mlsanity.com/paper/2010.05700/) (2020)
- [Sequence to Sequence -- Video to Text](https://mlsanity.com/paper/1505.00487/) (2015)
- [Style Transfer in Text: Exploration and Evaluation](https://mlsanity.com/paper/1711.06861/) (2017)
- [Disentangled Representation Learning for Non-Parallel Text Style Transfer](https://mlsanity.com/paper/1808.04339/) (2018)
- [Understanding Abuse: A Typology of Abusive Language Detection Subtasks](https://mlsanity.com/paper/1705.09899/) (2017)
- [Spatio-Temporal Graph for Video Captioning with Knowledge Distillation](https://mlsanity.com/paper/2003.13942/) (2020)
- [A Recipe For Arbitrary Text Style Transfer with Large Language Models](https://mlsanity.com/paper/2109.03910/) (2021)
- [Pre-training via Paraphrasing](https://mlsanity.com/paper/2006.15020/) (2020)
- [Say As You Wish: Fine-grained Control of Image Caption Generation with Abstract Scene Graphs](https://mlsanity.com/paper/2003.00387/) (2020)
- [Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation](https://mlsanity.com/paper/1905.05621/) (2019)

## Built on by (most cited first, 11 of 11)

- [Second Thoughts are Best: Learning to Re-Align With Human Values from Text Edits](https://mlsanity.com/paper/2301.00355/) (2023)
- [Boosting Distress Support Dialogue Responses with Motivational Interviewing Strategy](https://mlsanity.com/paper/2305.10195/) (2023)
- [STEER: Unified Style Transfer with Expert Reinforcement](https://mlsanity.com/paper/2311.07167/) (2023)
- [Distilling Text Style Transfer With Self-Explanation From LLMs](https://mlsanity.com/paper/2403.01106/) (2024)
- [DetoxLLM: A Framework for Detoxification with Explanations](https://mlsanity.com/paper/2402.15951/) (2024)
- [Prefix-Tuning Based Unsupervised Text Style Transfer](https://mlsanity.com/paper/2310.14599/) (2023)
- [Translating away Translationese without Parallel Data](https://mlsanity.com/paper/2310.18830/) (2023)
- [GPT-DETOX: An In-Context Learning-Based Paraphraser for Text Detoxification](https://mlsanity.com/paper/2404.03052/) (2024)
- [GradEscape: A Gradient-Based Evader Against AI-Generated Text Detectors](https://mlsanity.com/paper/2506.08188/) (2025)
- [One Agent to Serve All: a Lite-Adaptive Stylized AI Assistant for Millions of Multi-Style Official Accounts](https://mlsanity.com/paper/2509.17788/) (2025)
- [SimpleStyle: An Adaptable Style Transfer Approach](https://mlsanity.com/paper/2212.10498/) (2022)

## Abstract (arXiv)

The performance of existing text style transfer models is severely limited by the non-parallel datasets on which the models are trained. In non-parallel datasets, no direct mapping exists between sentences of the source and target style; the style transfer models thus only receive weak supervision of the target sentences during training, which often leads the model to discard too much style-independent information, or utterly fail to transfer the style. In this work, we propose LaMer, a novel text style transfer framework based on large-scale language models. LaMer first mines the roughly parallel expressions in the non-parallel datasets with scene graphs, and then employs MLE training, followed by imitation learning refinement, to leverage the intrinsic parallelism within the data. On two benchmark tasks (sentiment & formality transfer) and a newly proposed challenging task (political stance transfer), our model achieves qualitative advances in transfer accuracy, content preservation, and fluency. Further empirical and human evaluations demonstrate that our model not only makes training more efficient, but also generates more readable and diverse expressions than previous models.