# Deep Learning Approaches to Lexical Simplification: A Survey

arXiv: https://arxiv.org/abs/2305.12000 · HTML: https://mlsanity.com/paper/2305.12000/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 93 references that are in the map.
- First posted on arXiv 19 May 2023.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [MarIA: Spanish Language Models](https://mlsanity.com/paper/2107.07253/) (2021)
- [A Report on the Complex Word Identification Shared Task 2018](https://mlsanity.com/paper/1804.09132/) (2018)
- [A Word-Complexity Lexicon and A Neural Readability Ranking Model for Lexical Simplification](https://mlsanity.com/paper/1810.05754/) (2018)
- [CompLex: A New Corpus for Lexical Complexity Prediction from Likert Scale Data](https://mlsanity.com/paper/2003.07008/) (2020)
- [sense2vec - A Fast and Accurate Method for Word Sense Disambiguation In Neural Word Embeddings](https://mlsanity.com/paper/1511.06388/) (2015)
- [Findings of the TSAR-2022 Shared Task on Multilingual Lexical Simplification](https://mlsanity.com/paper/2302.02888/) (2023)
- [SemEval-2021 Task 1: Lexical Complexity Prediction](https://mlsanity.com/paper/2106.00473/) (2021)
- [Always Keep your Target in Mind: Studying Semantics and Improving Performance of Neural Lexical Substitution](https://mlsanity.com/paper/2206.11815/) (2022)
- [UniHD at TSAR-2022 Shared Task: Is Compute All We Need for Lexical Simplification?](https://mlsanity.com/paper/2301.01764/) (2023)
- [One Size Does Not Fit All: The Case for Personalised Word Complexity Models](https://mlsanity.com/paper/2205.02564/) (2022)
- [Chinese Lexical Simplification](https://mlsanity.com/paper/2010.07048/) (2020)
- [MANTIS at TSAR-2022 Shared Task: Improved Unsupervised Lexical Simplification with Pretrained Encoders](https://mlsanity.com/paper/2212.09855/) (2022)
- [One Size Does Not Fit All: Finding the Optimal Subword Sizes for FastText Models across Languages](https://mlsanity.com/paper/2102.02585/) (2021)
- [Lexical Complexity Prediction: An Overview](https://mlsanity.com/paper/2303.04851/) (2023)

## Built on by (most cited first, 4 of 4)

- [Evaluating Document Simplification: On the Importance of Separately Assessing Simplicity and Meaning Preservation](https://mlsanity.com/paper/2404.03278/) (2024)
- [Health Text Simplification: An Annotated Corpus for Digestive Cancer Education and Novel Strategies for Reinforcement Learning](https://mlsanity.com/paper/2401.15043/) (2024)
- [RALS: Resources and Baselines for Romanian Automatic Lexical Simplification](https://mlsanity.com/paper/2607.20078/) (2026)
- [Right at My Level: A Unified Multilingual Framework for Proficiency-Aware Text Simplification](https://mlsanity.com/paper/2604.05302/) (2026)

## Abstract (arXiv)

Lexical Simplification (LS) is the task of replacing complex for simpler words in a sentence whilst preserving the sentence's original meaning. LS is the lexical component of Text Simplification (TS) with the aim of making texts more accessible to various target populations. A past survey (Paetzold and Specia, 2017) has provided a detailed overview of LS. Since this survey, however, the AI/NLP community has been taken by storm by recent advances in deep learning, particularly with the introduction of large language models (LLM) and prompt learning. The high performance of these models sparked renewed interest in LS. To reflect these recent advances, we present a comprehensive survey of papers published between 2017 and 2023 on LS and its sub-tasks with a special focus on deep learning. We also present benchmark datasets for the future development of LS systems.