# Don't lose the message while paraphrasing: A study on content preserving style transfer

arXiv: https://arxiv.org/abs/2308.09055 · HTML: https://mlsanity.com/paper/2308.09055/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 32 references that are in the map.
- First posted on arXiv 17 Aug 2023.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [Neural Network Acceptability Judgments](https://mlsanity.com/paper/1805.12471/) (2018)
- [Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset](https://mlsanity.com/paper/1909.05855/) (2019)
- [Style Transfer from Non-Parallel Text by Cross-Alignment](https://mlsanity.com/paper/1705.09655/) (2017)
- [Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer](https://mlsanity.com/paper/1803.06535/) (2018)
- [Levenshtein Transformer](https://mlsanity.com/paper/1905.11006/) (2019)
- [Reformulating Unsupervised Style Transfer as Paraphrase Generation](https://mlsanity.com/paper/2010.05700/) (2020)
- [A Recipe For Arbitrary Text Style Transfer with Large Language Models](https://mlsanity.com/paper/2109.03910/) (2021)
- [Shakespearizing Modern Language Using Copy-Enriched Sequence-to-Sequence Models](https://mlsanity.com/paper/1707.01161/) (2017)
- ["Mask and Infill" : Applying Masked Language Model to Sentiment Transfer](https://mlsanity.com/paper/1908.08039/) (2019)
- [LEWIS: Levenshtein Editing for Unsupervised Text Style Transfer](https://mlsanity.com/paper/2105.08206/) (2021)
- [Parallel Data Augmentation for Formality Style Transfer](https://mlsanity.com/paper/2005.07522/) (2020)
- [Neural Machine Translation with Noisy Lexical Constraints](https://mlsanity.com/paper/1908.04664/) (2019)
- [Studying the role of named entities for content preservation in text style transfer](https://mlsanity.com/paper/2206.09676/) (2022)

## Built on by (most cited first, 25 of 25)

- [Controlled Text Generation via Language Model Arithmetic](https://mlsanity.com/paper/2311.14479/) (2023)
- [Value Augmented Sampling for Language Model Alignment and Personalization](https://mlsanity.com/paper/2405.06639/) (2024)
- [Threads of Subtlety: Detecting Machine-Generated Texts Through Discourse Motifs](https://mlsanity.com/paper/2402.10586/) (2024)
- [StyleRemix: Interpretable Authorship Obfuscation via Distillation and Perturbation of Style Elements](https://mlsanity.com/paper/2408.15666/) (2024)
- [Distilling Text Style Transfer With Self-Explanation From LLMs](https://mlsanity.com/paper/2403.01106/) (2024)
- [Out of Style: RAG's Fragility to Linguistic Variation](https://mlsanity.com/paper/2504.08231/) (2025)
- [Self-interpreting Adversarial Images](https://mlsanity.com/paper/2407.08970/) (2024)
- [Contextual Thompson Sampling via Generation of Missing Data](https://mlsanity.com/paper/2502.07064/) (2025)
- [Linguistic Comparison of AI- and Human-Written Responses to Online Mental Health Queries](https://mlsanity.com/paper/2504.09271/) (2025)
- [Interpersonal Theory of Suicide as a Lens to Examine Suicidal Ideation in Online Spaces](https://mlsanity.com/paper/2504.13277/) (2025)
- [Social Simulacra in the Wild: AI Agent Communities on Moltbook](https://mlsanity.com/paper/2603.16128/) (2026)
- [Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation](https://mlsanity.com/paper/2409.00696/) (2024)
- [Answer Bubbles: Information Exposure in AI-Mediated Search](https://mlsanity.com/paper/2603.16138/) (2026)
- [Steering Without Breaking: Mechanistically Informed Interventions for Discrete Diffusion Language Models](https://mlsanity.com/paper/2605.10971/) (2026)
- [Beyond checkmate: exploring the creative chokepoints in AI text](https://mlsanity.com/paper/2501.19301/) (2025)
- [Echoes of Automation: The Increasing Use of LLMs in Newsmaking](https://mlsanity.com/paper/2508.06445/) (2025)
- [Does AI Coaching Prepare us for Workplace Negotiations?](https://mlsanity.com/paper/2509.22545/) (2025)
- [GenCtrl -- A Formal Controllability Toolkit for Generative Models](https://mlsanity.com/paper/2601.05637/) (2026)
- [Not My Truce: Personality Differences in AI-Mediated Workplace Negotiation](https://mlsanity.com/paper/2604.00464/) (2026)
- [LLUMI: Improving LLM Writing Assistance for Mental Health Support with Online Community Feedback](https://mlsanity.com/paper/2605.30273/) (2026)
- [FAME-MT Dataset: Formality Awareness Made Easy for Machine Translation Purposes](https://mlsanity.com/paper/2405.11942/) (2024)
- [Empathy by Design: Aligning Large Language Models for Healthcare Dialogue](https://mlsanity.com/paper/2512.06097/) (2025)
- [A Test of Time: Predicting the Sustainable Success of Online Collaboration in Wikipedia](https://mlsanity.com/paper/2410.19150/) (2024)
- [Pareto-Guided Teacher Alignment for Fair Personalized Text Generation](https://mlsanity.com/paper/2606.10126/) (2026)
- [Trustworthy FinAInce: Unpacking How AI-Mediated Financial Advice is Judged](https://mlsanity.com/paper/2609.20989/) (2026)

## Abstract (arXiv)

Text style transfer techniques are gaining popularity in natural language processing allowing paraphrasing text in the required form: from toxic to neural, from formal to informal, from old to the modern English language, etc. Solving the task is not sufficient to generate some neural/informal/modern text, but it is important to preserve the original content unchanged. This requirement becomes even more critical in some applications such as style transfer of goal-oriented dialogues where the factual information shall be kept to preserve the original message, e.g. ordering a certain type of pizza to a certain address at a certain time. The aspect of content preservation is critical for real-world applications of style transfer studies, but it has received little attention. To bridge this gap we perform a comparison of various style transfer models on the example of the formality transfer domain. To perform a study of the content preservation abilities of various style transfer methods we create a parallel dataset of formal vs. informal task-oriented dialogues. The key difference between our dataset and the existing ones like GYAFC [17] is the presence of goal-oriented dialogues with predefined semantic slots essential to be kept during paraphrasing, e.g. named entities. This additional annotation allowed us to conduct a precise comparative study of several state-of-the-art techniques for style transfer. Another result of our study is a modification of the unsupervised method LEWIS [19] which yields a substantial improvement over the original method and all evaluated baselines on the proposed task.