# Towards Universality in Multilingual Text Rewriting

arXiv: https://arxiv.org/abs/2107.14749 · HTML: https://mlsanity.com/paper/2107.14749/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 39 references that are in the map.
- First posted on arXiv 30 Jul 2021.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [Adafactor: Adaptive Learning Rates with Sublinear Memory Cost](https://mlsanity.com/paper/1804.04235/) (2018)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation](https://mlsanity.com/paper/2004.11867/) (2020)
- [XFORMAL: A Benchmark for Multilingual Formality Style Transfer](https://mlsanity.com/paper/2104.04108/) (2021)
- [Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation](https://mlsanity.com/paper/2005.04816/) (2020)
- [CoCon: A Self-Supervised Approach for Controlled Text Generation](https://mlsanity.com/paper/2006.03535/) (2020)
- [When and Why is Unsupervised Neural Machine Translation Useless?](https://mlsanity.com/paper/2004.10581/) (2020)
- [Multi-Task Neural Models for Translating Between Styles Within and Across Languages](https://mlsanity.com/paper/1806.04357/) (2018)
- [Extreme Adaptation for Personalized Neural Machine Translation](https://mlsanity.com/paper/1805.01817/) (2018)
- [A Multilingual View of Unsupervised Machine Translation](https://mlsanity.com/paper/2002.02955/) (2020)
- [Harnessing Multilinguality in Unsupervised Machine Translation for Rare Languages](https://mlsanity.com/paper/2009.11201/) (2020)
- [Machine Translation into Low-resource Language Varieties](https://mlsanity.com/paper/2106.06797/) (2021)
- [Reference Language based Unsupervised Neural Machine Translation](https://mlsanity.com/paper/2004.02127/) (2020)
- [Neural Machine Translation into Language Varieties](https://mlsanity.com/paper/1811.01064/) (2018)
- [Towards Personalised and Document-level Machine Translation of Dialogue](https://mlsanity.com/paper/2102.10979/) (2021)
- [Unsupervised Learning of Style-sensitive Word Vectors](https://mlsanity.com/paper/1805.05581/) (2018)

## Built on by (most cited first, 7 of 7)

- [The unreasonable effectiveness of few-shot learning for machine translation](https://mlsanity.com/paper/2302.01398/) (2023)
- [Few-shot Controllable Style Transfer for Low-Resource Multilingual Settings](https://mlsanity.com/paper/2110.07385/) (2021)
- [FRMT: A Benchmark for Few-Shot Region-Aware Machine Translation](https://mlsanity.com/paper/2210.00193/) (2022)
- [RAMP: Retrieval and Attribute-Marking Enhanced Prompting for Attribute-Controlled Translation](https://mlsanity.com/paper/2305.17131/) (2023)
- [Towards Style Alignment in Cross-Cultural Translation](https://mlsanity.com/paper/2507.00216/) (2025)
- [Comparing Styles across Languages: A Cross-Cultural Exploration of Politeness](https://mlsanity.com/paper/2310.07135/) (2023)
- [A Survey of Text Style Transfer: Applications and Ethical Implications](https://mlsanity.com/paper/2407.16737/) (2024)

## Abstract (arXiv)

In this work, we take the first steps towards building a universal rewriter: a model capable of rewriting text in any language to exhibit a wide variety of attributes, including styles and languages, while preserving as much of the original semantics as possible. In addition to obtaining state-of-the-art results on unsupervised translation, we also demonstrate the ability to do zero-shot sentiment transfer in non-English languages using only English exemplars for sentiment. We then show that our model is able to modify multiple attributes at once, for example adjusting both language and sentiment jointly. Finally, we show that our model is capable of performing zero-shot formality-sensitive translation.