# Learning to Substitute Words with Model-based Score Ranking

arXiv: https://arxiv.org/abs/2502.05933 · HTML: https://mlsanity.com/paper/2502.05933/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 35 references that are in the map.
- First posted on arXiv 9 Feb 2025.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [BARTScore: Evaluating Generated Text as Text Generation](https://mlsanity.com/paper/2106.11520/) (2021)
- [BRIO: Bringing Order to Abstractive Summarization](https://mlsanity.com/paper/2203.16804/) (2022)
- [On Learning to Summarize with Large Language Models as References](https://mlsanity.com/paper/2305.14239/) (2023)
- [InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward Modeling](https://mlsanity.com/paper/2402.09345/) (2024)
- [Improving Machine Translation with Human Feedback: An Exploration of Quality Estimation as a Reward Model](https://mlsanity.com/paper/2401.12873/) (2024)
- [Toward Human-Like Evaluation for Natural Language Generation with Error Analysis](https://mlsanity.com/paper/2212.10179/) (2022)
- [Improving Factuality of Abstractive Summarization via Contrastive Reward Learning](https://mlsanity.com/paper/2307.04507/) (2023)
- [LexSubCon: Integrating Knowledge from Lexical Resources into Contextual Embeddings for Lexical Substitution](https://mlsanity.com/paper/2107.05132/) (2021)
- [Smart Word Suggestions for Writing Assistance](https://mlsanity.com/paper/2305.09975/) (2023)

## Built on by (most cited first, 10 of 10)

- [Localizing Credit at the Divergence: Path-Conditioned Self-Distillation for LLM Reasoning](https://mlsanity.com/paper/2606.15576/) (2026)
- [PERM: Psychology-grounded Empathetic Reward Modeling for Large Language Models](https://mlsanity.com/paper/2601.10532/) (2026)
- [Learning to Control Summaries with Score Ranking](https://mlsanity.com/paper/2604.17197/) (2026)
- [RCBSF: A Multi-Agent Framework for Automated Contract Revision via Stackelberg Game](https://mlsanity.com/paper/2604.10740/) (2026)
- [DimMem: Dimensional Structuring for Efficient Long-Term Agent Memory](https://mlsanity.com/paper/2605.15759/) (2026)
- [Who Wrote This Line? Evaluating the Detection of LLM-Generated Classical Chinese Poetry](https://mlsanity.com/paper/2604.10101/) (2026)
- [A Heuristic Perspective on Debiasing Language Models](https://mlsanity.com/paper/2608.00622/) (2026)
- [Surfacing the Unsaid: CUE-Bench for Affective Stance in Chinese Discourse](https://mlsanity.com/paper/2608.10810/) (2026)
- [Do LLMs Capture Embodied Cognition and Cultural Variation? Cross-Linguistic Evidence from Demonstratives](https://mlsanity.com/paper/2604.25423/) (2026)
- [Too Good to Be Real? Diagnosing and Reducing the Gap Between AI Preference and Real User Engagement](https://mlsanity.com/paper/2609.18282/) (2026)

## Abstract (arXiv)

Smart word substitution aims to enhance sentence quality by improving word choices; however current benchmarks rely on human-labeled data. Since word choices are inherently subjective, ground-truth word substitutions generated by a small group of annotators are often incomplete and likely not generalizable. To circumvent this issue, we instead employ a model-based score (BARTScore) to quantify sentence quality, thus forgoing the need for human annotations. Specifically, we use this score to define a distribution for each word substitution, allowing one to test whether a substitution is statistically superior relative to others. In addition, we propose a loss function that directly optimizes the alignment between model predictions and sentence scores, while also enhancing the overall quality score of a substitution. Crucially, model learning no longer requires human labels, thus avoiding the cost of annotation while maintaining the quality of the text modified with substitutions. Experimental results show that the proposed approach outperforms both masked language models (BERT, BART) and large language models (GPT-4, LLaMA). The source code is available at https://github.com/Hyfred/Substitute-Words-with-Ranking.