# Semi-supervised Formality Style Transfer using Language Model Discriminator and Mutual Information Maximization

arXiv: https://arxiv.org/abs/2010.05090 · HTML: https://mlsanity.com/paper/2010.05090/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Oct 2020.

## Built on by (most cited first, 19 of 19)

- [Meet Your Favorite Character: Open-domain Chatbot Mimicking Fictional Characters with only a Few Utterances](https://mlsanity.com/paper/2204.10825/) (2022)
- [Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer](https://mlsanity.com/paper/2105.06947/) (2021)
- [APPDIA: A Discourse-aware Transformer-based Style Transfer Model for Offensive Social Media Conversations](https://mlsanity.com/paper/2209.08207/) (2022)
- [MVP: Multi-task Supervised Pre-training for Natural Language Generation](https://mlsanity.com/paper/2206.12131/) (2022)
- [Semi-Supervised Formality Style Transfer with Consistency Training](https://mlsanity.com/paper/2203.13620/) (2022)
- [Multidimensional Evaluation for Text Style Transfer Using ChatGPT](https://mlsanity.com/paper/2304.13462/) (2023)
- [Distilling Text Style Transfer With Self-Explanation From LLMs](https://mlsanity.com/paper/2403.01106/) (2024)
- [Generic resources are what you need: Style transfer tasks without task-specific parallel training data](https://mlsanity.com/paper/2109.04543/) (2021)
- [A Review of Human Emotion Synthesis Based on Generative Technology](https://mlsanity.com/paper/2412.07116/) (2024)
- [Don't Take This Out of Context! On the Need for Contextual Models and Evaluations for Stylistic Rewriting](https://mlsanity.com/paper/2305.14755/) (2023)
- [From Theories on Styles to their Transfer in Text: Bridging the Gap with a Hierarchical Survey](https://mlsanity.com/paper/2110.15871/) (2021)
- [The Power of Fragmentation: A Hierarchical Transformer Model for Structural Segmentation in Symbolic Music Generation](https://mlsanity.com/paper/2205.08579/) (2022)
- [Audience-Centric Natural Language Generation via Style Infusion](https://mlsanity.com/paper/2301.10283/) (2023)
- [Reducing Sequence Length by Predicting Edit Operations with Large Language Models](https://mlsanity.com/paper/2305.11862/) (2023)
- [VQA-GEN: A Visual Question Answering Benchmark for Domain Generalization](https://mlsanity.com/paper/2311.00807/) (2023)
- [Exploiting Social Media Content for Self-Supervised Style Transfer](https://mlsanity.com/paper/2205.08814/) (2022)
- [He Said, She Said: Style Transfer for Shifting the Perspective of Dialogues](https://mlsanity.com/paper/2210.15462/) (2022)
- [Learning from Bootstrapping and Stepwise Reinforcement Reward: A Semi-Supervised Framework for Text Style Transfer](https://mlsanity.com/paper/2205.09324/) (2022)
- [Learning a Formality-Aware Japanese Sentence Representation](https://mlsanity.com/paper/2301.07209/) (2023)

## Abstract (arXiv)

Formality style transfer is the task of converting informal sentences to grammatically-correct formal sentences, which can be used to improve performance of many downstream NLP tasks. In this work, we propose a semi-supervised formality style transfer model that utilizes a language model-based discriminator to maximize the likelihood of the output sentence being formal, which allows us to use maximization of token-level conditional probabilities for training. We further propose to maximize mutual information between source and target styles as our training objective instead of maximizing the regular likelihood that often leads to repetitive and trivial generated responses. Experiments showed that our model outperformed previous state-of-the-art baselines significantly in terms of both automated metrics and human judgement. We further generalized our model to unsupervised text style transfer task, and achieved significant improvements on two benchmark sentiment style transfer datasets.