# ParaLS: Lexical Substitution via Pretrained Paraphraser

arXiv: https://arxiv.org/abs/2305.08146 · HTML: https://mlsanity.com/paper/2305.08146/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 45 references that are in the map.
- First posted on arXiv 14 May 2023.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [BARTScore: Evaluating Generated Text as Text Generation](https://mlsanity.com/paper/2106.11520/) (2021)
- [NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristics](https://mlsanity.com/paper/2112.08726/) (2021)
- [Contextualized Perturbation for Textual Adversarial Attack](https://mlsanity.com/paper/2009.07502/) (2020)
- [Noisy Parallel Approximate Decoding for Conditional Recurrent Language Model](https://mlsanity.com/paper/1605.03835/) (2016)
- [Swords: A Benchmark for Lexical Substitution with Improved Data Coverage and Quality](https://mlsanity.com/paper/2106.04102/) (2021)
- [ConRPG: Paraphrase Generation using Contexts as Regularizer](https://mlsanity.com/paper/2109.00363/) (2021)
- [Chinese Lexical Simplification](https://mlsanity.com/paper/2010.07048/) (2020)
- [LexSubCon: Integrating Knowledge from Lexical Resources into Contextual Embeddings for Lexical Substitution](https://mlsanity.com/paper/2107.05132/) (2021)
- [A Comparative Study of Lexical Substitution Approaches based on Neural Language Models](https://mlsanity.com/paper/2006.00031/) (2020)
- [Improving Contextual Representation with Gloss Regularized Pre-training](https://mlsanity.com/paper/2205.06603/) (2022)

## Built on by (most cited first, 6 of 6)

- [An LLM-Enhanced Adversarial Editing System for Lexical Simplification](https://mlsanity.com/paper/2402.14704/) (2024)
- [Progressive Document-level Text Simplification via Large Language Models](https://mlsanity.com/paper/2501.03857/) (2025)
- [Comparing Feature-based and Context-aware Approaches to PII Generalization Level Prediction](https://mlsanity.com/paper/2407.02837/) (2024)
- [Sentence Simplification Using Paraphrase Corpus for Initialization](https://mlsanity.com/paper/2305.19754/) (2023)
- [Lexical Substitution is not Synonym Substitution: On the Importance of Producing Contextually Relevant Word Substitutes](https://mlsanity.com/paper/2502.04173/) (2025)
- [Unsupervised Candidate Ranking for Lexical Substitution via Holistic Sentence Semantics](https://mlsanity.com/paper/2509.11513/) (2025)

## Abstract (arXiv)

Lexical substitution (LS) aims at finding appropriate substitutes for a target word in a sentence. Recently, LS methods based on pretrained language models have made remarkable progress, generating potential substitutes for a target word through analysis of its contextual surroundings. However, these methods tend to overlook the preservation of the sentence's meaning when generating the substitutes. This study explores how to generate the substitute candidates from a paraphraser, as the generated paraphrases from a paraphraser contain variations in word choice and preserve the sentence's meaning. Since we cannot directly generate the substitutes via commonly used decoding strategies, we propose two simple decoding strategies that focus on the variations of the target word during decoding. Experimental results show that our methods outperform state-of-the-art LS methods based on pre-trained language models on three benchmarks.