# XFORMAL: A Benchmark for Multilingual Formality Style Transfer

arXiv: https://arxiv.org/abs/2104.04108 · HTML: https://mlsanity.com/paper/2104.04108/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 68 references that are in the map.
- First posted on arXiv 8 Apr 2021.

## Builds on (most cited first, 30 of 41)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [XNLI: Evaluating Cross-lingual Sentence Representations](https://mlsanity.com/paper/1809.05053/) (2018)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [On the Cross-lingual Transferability of Monolingual Representations](https://mlsanity.com/paper/1910.11856/) (2019)
- [Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering](https://mlsanity.com/paper/1602.01585/) (2016)
- [Texygen: A Benchmarking Platform for Text Generation Models](https://mlsanity.com/paper/1802.01886/) (2018)
- [Toward Controlled Generation of Text](https://mlsanity.com/paper/1703.00955/) (2017)
- [Style Transfer from Non-Parallel Text by Cross-Alignment](https://mlsanity.com/paper/1705.09655/) (2017)
- [Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer](https://mlsanity.com/paper/1803.06535/) (2018)
- [Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer](https://mlsanity.com/paper/1804.06437/) (2018)
- [Reformulating Unsupervised Style Transfer as Paraphrase Generation](https://mlsanity.com/paper/2010.05700/) (2020)
- [Style Transfer Through Back-Translation](https://mlsanity.com/paper/1804.09000/) (2018)
- [BLEU might be Guilty but References are not Innocent](https://mlsanity.com/paper/2004.06063/) (2020)
- [Getting Gender Right in Neural Machine Translation](https://mlsanity.com/paper/1909.05088/) (2019)
- [A Probabilistic Formulation of Unsupervised Text Style Transfer](https://mlsanity.com/paper/2002.03912/) (2020)
- [A Dual Reinforcement Learning Framework for Unsupervised Text Style Transfer](https://mlsanity.com/paper/1905.10060/) (2019)
- [Evaluating Style Transfer for Text](https://mlsanity.com/paper/1904.02295/) (2019)
- [Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach](https://mlsanity.com/paper/1805.05181/) (2018)
- [Personalized Machine Translation: Preserving Original Author Traits](https://mlsanity.com/paper/1610.05461/) (2016)
- [Content preserving text generation with attribute controls](https://mlsanity.com/paper/1811.01135/) (2018)
- [Expertise Style Transfer: A New Task Towards Better Communication between Experts and Laymen](https://mlsanity.com/paper/2005.00701/) (2020)
- [Parallel Data Augmentation for Formality Style Transfer](https://mlsanity.com/paper/2005.07522/) (2020)
- [Reinforcement Learning Based Text Style Transfer without Parallel Training Corpus](https://mlsanity.com/paper/1903.10671/) (2019)
- [Neural Machine Translation Decoding with Terminology Constraints](https://mlsanity.com/paper/1805.03750/) (2018)
- [IMaT: Unsupervised Text Attribute Transfer via Iterative Matching and Translation](https://mlsanity.com/paper/1901.11333/) (2019)
- [Controlling Text Complexity in Neural Machine Translation](https://mlsanity.com/paper/1911.00835/) (2019)
- [Multi-Task Neural Models for Translating Between Styles Within and Across Languages](https://mlsanity.com/paper/1806.04357/) (2018)
- [Filling Gender & Number Gaps in Neural Machine Translation with Black-box Context Injection](https://mlsanity.com/paper/1903.03467/) (2019)

## Built on by (most cited first, 30 of 52)

- [Multi-property Steering of Large Language Models with Dynamic Activation Composition](https://mlsanity.com/paper/2406.17563/) (2024)
- [NEWTS: A Corpus for News Topic-Focused Summarization](https://mlsanity.com/paper/2205.15661/) (2022)
- [Revisiting non-English Text Simplification: A Unified Multilingual Benchmark](https://mlsanity.com/paper/2305.15678/) (2023)
- [Evaluating the Evaluation Metrics for Style Transfer: A Case Study in Multilingual Formality Transfer](https://mlsanity.com/paper/2110.10668/) (2021)
- [Few-shot Controllable Style Transfer for Low-Resource Multilingual Settings](https://mlsanity.com/paper/2110.07385/) (2021)
- [TyDiP: A Dataset for Politeness Classification in Nine Typologically Diverse Languages](https://mlsanity.com/paper/2211.16496/) (2022)
- [IT5: Text-to-text Pretraining for Italian Language Understanding and Generation](https://mlsanity.com/paper/2203.03759/) (2022)
- [A Review of Human Evaluation for Style Transfer](https://mlsanity.com/paper/2106.04747/) (2021)
- [ParaGuide: Guided Diffusion Paraphrasers for Plug-and-Play Textual Style Transfer](https://mlsanity.com/paper/2308.15459/) (2023)
- [Can AI Be as Creative as Humans?](https://mlsanity.com/paper/2401.01623/) (2024)
- [Advanced Natural-based interaction for the ITAlian language: LLaMAntino-3-ANITA](https://mlsanity.com/paper/2405.07101/) (2024)
- [Exploring Methods for Cross-lingual Text Style Transfer: The Case of Text Detoxification](https://mlsanity.com/paper/2311.13937/) (2023)
- [A Call for Standardization and Validation of Text Style Transfer Evaluation](https://mlsanity.com/paper/2306.00539/) (2023)
- [MultiParaDetox: Extending Text Detoxification with Parallel Data to New Languages](https://mlsanity.com/paper/2404.02037/) (2024)
- [Camoscio: an Italian Instruction-tuned LLaMA](https://mlsanity.com/paper/2307.16456/) (2023)
- [mEdIT: Multilingual Text Editing via Instruction Tuning](https://mlsanity.com/paper/2402.16472/) (2024)
- [Semi-Supervised Formality Style Transfer with Consistency Training](https://mlsanity.com/paper/2203.13620/) (2022)
- [Human Judgement as a Compass to Navigate Automatic Metrics for Formality Transfer](https://mlsanity.com/paper/2204.07549/) (2022)
- [Multilingual and Explainable Text Detoxification with Parallel Corpora](https://mlsanity.com/paper/2412.11691/) (2024)
- [Detecting Text Formality: A Study of Text Classification Approaches](https://mlsanity.com/paper/2204.08975/) (2022)
- [GPTs Are Multilingual Annotators for Sequence Generation Tasks](https://mlsanity.com/paper/2402.05512/) (2024)
- [RAMP: Retrieval and Attribute-Marking Enhanced Prompting for Attribute-Controlled Translation](https://mlsanity.com/paper/2305.17131/) (2023)
- [Learning to Generate Text in Arbitrary Writing Styles](https://mlsanity.com/paper/2312.17242/) (2023)
- [Multilingual Text Style Transfer: Datasets & Models for Indian Languages](https://mlsanity.com/paper/2405.20805/) (2024)
- [Multilingual Pre-training with Language and Task Adaptation for Multilingual Text Style Transfer](https://mlsanity.com/paper/2203.08552/) (2022)
- [NAP^2: A Benchmark for Naturalness and Privacy-Preserving Text Rewriting by Learning from Human](https://mlsanity.com/paper/2406.03749/) (2024)
- [Towards Universality in Multilingual Text Rewriting](https://mlsanity.com/paper/2107.14749/) (2021)
- [CrisisLTLSum: A Benchmark for Local Crisis Event Timeline Extraction and Summarization](https://mlsanity.com/paper/2210.14190/) (2022)
- [A Block Metropolis-Hastings Sampler for Controllable Energy-based Text Generation](https://mlsanity.com/paper/2312.04510/) (2023)
- [From Theories on Styles to their Transfer in Text: Bridging the Gap with a Hierarchical Survey](https://mlsanity.com/paper/2110.15871/) (2021)

## Abstract (arXiv)

We take the first step towards multilingual style transfer by creating and releasing XFORMAL, a benchmark of multiple formal reformulations of informal text in Brazilian Portuguese, French, and Italian. Results on XFORMAL suggest that state-of-the-art style transfer approaches perform close to simple baselines, indicating that style transfer is even more challenging when moving multilingual.