# Unsupervised Lexical Simplification with Context Augmentation

arXiv: https://arxiv.org/abs/2311.00310 · HTML: https://mlsanity.com/paper/2311.00310/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 24 references that are in the map.
- First posted on arXiv 1 Nov 2023.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [Spanish Pre-trained BERT Model and Evaluation Data](https://mlsanity.com/paper/2308.02976/) (2023)
- [Findings of the TSAR-2022 Shared Task on Multilingual Lexical Simplification](https://mlsanity.com/paper/2302.02888/) (2023)
- [Lexical Simplification Benchmarks for English, Portuguese, and Spanish](https://mlsanity.com/paper/2209.05301/) (2022)
- [Swords: A Benchmark for Lexical Substitution with Improved Data Coverage and Quality](https://mlsanity.com/paper/2106.04102/) (2021)
- [UniHD at TSAR-2022 Shared Task: Is Compute All We Need for Lexical Simplification?](https://mlsanity.com/paper/2301.01764/) (2023)
- [LSBert: A Simple Framework for Lexical Simplification](https://mlsanity.com/paper/2006.14939/) (2020)
- [MANTIS at TSAR-2022 Shared Task: Improved Unsupervised Lexical Simplification with Pretrained Encoders](https://mlsanity.com/paper/2212.09855/) (2022)
- [ALEXSIS-PT: A New Resource for Portuguese Lexical Simplification](https://mlsanity.com/paper/2209.09034/) (2022)

## Abstract (arXiv)

We propose a new unsupervised lexical simplification method that uses only monolingual data and pre-trained language models. Given a target word and its context, our method generates substitutes based on the target context and also additional contexts sampled from monolingual data. We conduct experiments in English, Portuguese, and Spanish on the TSAR-2022 shared task, and show that our model substantially outperforms other unsupervised systems across all languages. We also establish a new state-of-the-art by ensembling our model with GPT-3.5. Lastly, we evaluate our model on the SWORDS lexical substitution data set, achieving a state-of-the-art result.