# MUSS: Multilingual Unsupervised Sentence Simplification by Mining Paraphrases

arXiv: https://arxiv.org/abs/2005.00352 · HTML: https://mlsanity.com/paper/2005.00352/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 May 2020.

## Built on by (most cited first, 15 of 15)

- [Text Simplification by Tagging](https://mlsanity.com/paper/2103.05070/) (2021)
- [Control Prefixes for Parameter-Efficient Text Generation](https://mlsanity.com/paper/2110.08329/) (2021)
- [Multitasking Framework for Unsupervised Simple Definition Generation](https://mlsanity.com/paper/2203.12926/) (2022)
- [Rethinking Automatic Evaluation in Sentence Simplification](https://mlsanity.com/paper/2104.07560/) (2021)
- [An Unsupervised Method for Building Sentence Simplification Corpora in Multiple Languages](https://mlsanity.com/paper/2109.00165/) (2021)
- [Learning to Paraphrase Sentences to Different Complexity Levels](https://mlsanity.com/paper/2308.02226/) (2023)
- [Investigating Large Language Models and Control Mechanisms to Improve Text Readability of Biomedical Abstracts](https://mlsanity.com/paper/2309.13202/) (2023)
- [Exploiting Summarization Data to Help Text Simplification](https://mlsanity.com/paper/2302.07124/) (2023)
- [Exploiting Social Media Content for Self-Supervised Style Transfer](https://mlsanity.com/paper/2205.08814/) (2022)
- [Cognitive Simplification Operations Improve Text Simplification](https://mlsanity.com/paper/2211.08825/) (2022)
- [Mitigating Data Scarcity for Large Language Models](https://mlsanity.com/paper/2302.01806/) (2023)
- [SimpleBERT: A Pre-trained Model That Learns to Generate Simple Words](https://mlsanity.com/paper/2204.07779/) (2022)
- [Large Language Models for Biomedical Text Simplification: Promising But Not There Yet](https://mlsanity.com/paper/2408.03871/) (2024)
- [Noisy Channel for Automatic Text Simplification](https://mlsanity.com/paper/2211.03152/) (2022)
- [Sentence Simplification Using Paraphrase Corpus for Initialization](https://mlsanity.com/paper/2305.19754/) (2023)

## Abstract (arXiv)

Progress in sentence simplification has been hindered by a lack of labeled parallel simplification data, particularly in languages other than English. We introduce MUSS, a Multilingual Unsupervised Sentence Simplification system that does not require labeled simplification data. MUSS uses a novel approach to sentence simplification that trains strong models using sentence-level paraphrase data instead of proper simplification data. These models leverage unsupervised pretraining and controllable generation mechanisms to flexibly adjust attributes such as length and lexical complexity at inference time. We further present a method to mine such paraphrase data in any language from Common Crawl using semantic sentence embeddings, thus removing the need for labeled data. We evaluate our approach on English, French, and Spanish simplification benchmarks and closely match or outperform the previous best supervised results, despite not using any labeled simplification data. We push the state of the art further by incorporating labeled simplification data.