# Enhancing Content Preservation in Text Style Transfer Using Reverse Attention and Conditional Layer Normalization arXiv: https://arxiv.org/abs/2108.00449 ยท HTML: https://mlsanity.com/paper/2108.00449/ ## Key facts - Cited by 20 papers in the map (data through 14 Oct 2026). - Builds on 19 of its 24 references that are in the map. - First posted on arXiv 1 Aug 2021. ## Builds on (most cited first, 19 of 19) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019) - [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014) - [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016) - [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016) - [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014) - [A Learned Representation For Artistic Style](https://mlsanity.com/paper/1610.07629/) (2016) - [Toward Controlled Generation of Text](https://mlsanity.com/paper/1703.00955/) (2017) - [Style Transfer from Non-Parallel Text by Cross-Alignment](https://mlsanity.com/paper/1705.09655/) (2017) - [Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer](https://mlsanity.com/paper/1804.06437/) (2018) - [Style Transfer Through Back-Translation](https://mlsanity.com/paper/1804.09000/) (2018) - [Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation](https://mlsanity.com/paper/1905.05621/) (2019) - [A Probabilistic Formulation of Unsupervised Text Style Transfer](https://mlsanity.com/paper/2002.03912/) (2020) - [Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach](https://mlsanity.com/paper/1805.05181/) (2018) - ["Mask and Infill" : Applying Masked Language Model to Sentiment Transfer](https://mlsanity.com/paper/1908.08039/) (2019) - [Controllable Unsupervised Text Attribute Transfer via Editing Entangled Latent Representation](https://mlsanity.com/paper/1905.12926/) (2019) - [Revision in Continuous Space: Unsupervised Text Style Transfer without Adversarial Learning](https://mlsanity.com/paper/1905.12304/) (2019) - [FAKTA: An Automatic End-to-End Fact Checking System](https://mlsanity.com/paper/1906.04164/) (2019) ## Built on by (most cited first, 20 of 20) - [Authors' Values and Attitudes Towards AI-bridged Scalable Personalization of Creative Language Arts](https://mlsanity.com/paper/2403.00439/) (2024) - [Human Judgement as a Compass to Navigate Automatic Metrics for Formality Transfer](https://mlsanity.com/paper/2204.07549/) (2022) - [Non-Parallel Text Style Transfer with Self-Parallel Supervision](https://mlsanity.com/paper/2204.08123/) (2022) - [CAT-LLM: Style-enhanced Large Language Models with Text Style Definition for Chinese Article-style Transfer](https://mlsanity.com/paper/2401.05707/) (2024) - [StoryTrans: Non-Parallel Story Author-Style Transfer with Discourse Representations and Content Enhancing](https://mlsanity.com/paper/2208.13423/) (2022) - [Towards Robust and Semantically Organised Latent Representations for Unsupervised Text Style Transfer](https://mlsanity.com/paper/2205.02309/) (2022) - [Prefix-Tuning Based Unsupervised Text Style Transfer](https://mlsanity.com/paper/2310.14599/) (2023) - [From Theories on Styles to their Transfer in Text: Bridging the Gap with a Hierarchical Survey](https://mlsanity.com/paper/2110.15871/) (2021) - [Text Style Transfer: An Introductory Overview](https://mlsanity.com/paper/2407.14822/) (2024) - [StyleFlow: Disentangle Latent Representations via Normalizing Flow for Unsupervised Text Style Transfer](https://mlsanity.com/paper/2212.09670/) (2022) - [(Psycho-)Linguistic Features Meet Transformer Models for Improved Explainable and Controllable Text Simplification](https://mlsanity.com/paper/2212.09848/) (2022) - [SC2: Towards Enhancing Content Preservation and Style Consistency in Long Text Style Transfer](https://mlsanity.com/paper/2406.04578/) (2024) - [Replacing Language Model for Style Transfer](https://mlsanity.com/paper/2211.07343/) (2022) - [ECQED: Emotion-Cause Quadruple Extraction in Dialogs](https://mlsanity.com/paper/2306.03969/) (2023) - [Studying the role of named entities for content preservation in text style transfer](https://mlsanity.com/paper/2206.09676/) (2022) - [Few-Shot Domain Adaptation for Charge Prediction on Unprofessional Descriptions](https://mlsanity.com/paper/2309.17313/) (2023) - [Test-Time-Matching: Decouple Personality, Memory, and Linguistic Style in LLM-based Role-Playing Language Agent](https://mlsanity.com/paper/2507.16799/) (2025) - [Coupling Local Context and Global Semantic Prototypes via a Hierarchical Architecture for Rhetorical Roles Labeling](https://mlsanity.com/paper/2603.03856/) (2026) - [On Text Style Transfer via Style Masked Language Models](https://mlsanity.com/paper/2210.06394/) (2022) - [HyperStyler: Low-resource Authorship Style Transfer via Context-aware Style Navigation and Hypernetworks](https://mlsanity.com/paper/2609.02772/) (2026) ## Abstract (arXiv) Text style transfer aims to alter the style (e.g., sentiment) of a sentence while preserving its content. A common approach is to map a given sentence to content representation that is free of style, and the content representation is fed to a decoder with a target style. Previous methods in filtering style completely remove tokens with style at the token level, which incurs the loss of content information. In this paper, we propose to enhance content preservation by implicitly removing the style information of each token with reverse attention, and thereby retain the content. Furthermore, we fuse content information when building the target style representation, making it dynamic with respect to the content. Our method creates not only style-independent content representation, but also content-dependent style representation in transferring style. Empirical results show that our method outperforms the state-of-the-art baselines by a large margin in terms of content preservation. In addition, it is also competitive in terms of style transfer accuracy and fluency.