# Contextual Text Denoising with Masked Language Models

arXiv: https://arxiv.org/abs/1910.14080 · HTML: https://mlsanity.com/paper/1910.14080/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2019.

## Built on by (most cited first, 8 of 8)

- [A Methodology for Generative Spelling Correction via Natural Spelling Errors Emulation across Multiple Domains and Languages](https://mlsanity.com/paper/2308.09435/) (2023)
- [Robust Learning for Text Classification with Multi-source Noise Simulation and Hard Example Mining](https://mlsanity.com/paper/2107.07113/) (2021)
- [Rank-Aware Negative Training for Semi-Supervised Text Classification](https://mlsanity.com/paper/2306.07621/) (2023)
- [A Comparative Analysis of Noise Reduction Methods in Sentiment Analysis on Noisy Bangla Texts](https://mlsanity.com/paper/2401.14360/) (2024)
- [Robust Embeddings Via Distributions](https://mlsanity.com/paper/2104.08420/) (2021)
- [Frustratingly Simple Prompting-based Text Denoising](https://mlsanity.com/paper/2402.15931/) (2024)
- [When LLM Agents Meet Graph Optimization: An Automated Data Quality Improvement Approach](https://mlsanity.com/paper/2510.08952/) (2025)
- [OpenRTAG: A Comprehensive Benchmark for Robust Text-Attributed Graph Learning under Data Quality Degradation](https://mlsanity.com/paper/2607.19108/) (2026)

## Abstract (arXiv)

Recently, with the help of deep learning models, significant advances have been made in different Natural Language Processing (NLP) tasks. Unfortunately, state-of-the-art models are vulnerable to noisy texts. We propose a new contextual text denoising algorithm based on the ready-to-use masked language model. The proposed algorithm does not require retraining of the model and can be integrated into any NLP system without additional training on paired cleaning training data. We evaluate our method under synthetic noise and natural noise and show that the proposed algorithm can use context information to correct noise text and improve the performance of noisy inputs in several downstream tasks.