mlsanityData to 7 Oct 2026

Spelling Correction with Denoising Transformer

arXiv 2105.05977 ↗First posted 12 May 2021cs.CL cs.AI cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

We present a novel method of performing spelling correction on short input strings, such as search queries or individual words. At its core lies a procedure for generating artificial typos which closely follow the error patterns manifested by humans. This procedure is used to train the production spelling correction model based on a transformer architecture. This model is currently served in the HubSpot product search. We show that our approach to typo generation is superior to the widespread practice of adding noise, which ignores human patterns. We also demonstrate how our approach may be extended to resource-scarce settings and train spelling correction models for Arabic, Greek, Russian, and Setswana languages, without using any labeled data.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 13 of its 43 references that are in the map. First posted on arXiv 12 May 2021.

Builds on

13 of 43 references.

References · 13

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  3. 2019Gmail Smart Compose: Real-Time Assisted Writingcited by 97
  4. 2018MaskGAN: Better Text Generation via Filling in the______cited by 89
  5. 2018Tensor2Tensor for Neural Machine Translationcited by 85
  6. 2019A Neural Grammatical Error Correction System Built On Better Pre-training and Sequential Transfer Learningcited by 26
  7. 2014LABR: A Large Scale Arabic Sentiment Analysis Benchmarkcited by 22
  8. 2013Complexity of Word Collocation Networks: A Preliminary Structural Analysiscited by 17
  9. 2019A Focus on Neural Machine Translation for African Languagescited by 15
  10. 2019Denoising based Sequence-to-Sequence Pre-training for Text Generationcited by 15
  11. 2017Neural Networks for Text Correction and Completion in Keyboard Decodingcited by 12
  12. 2017Artificial Error Generation with Machine Translation and Syntactic Patternscited by 11
1 more
  1. 2017Spelling Correction as a Foreign Languagecited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 2; 2023: 3; 2024: 2; 2025: 0; 2026: 1.

9 papers.

  1. 2022Correcting diacritics and typos with a ByT5 transformer modelcited by 5
  2. 2023A Methodology for Generative Spelling Correction via Natural Spelling Errors Emulation across Multiple Domains and Languagescited by 5
  3. 2021Correcting Arabic Soft Spelling Mistakes using BiLSTM-based Machine Learningcited by 3
  4. 2022DPCSpell: A Transformer-based Detector-Purificator-Corrector Framework for Spelling Error Correction of Bangla and Resource Scarce Indic Languagescited by 3
  5. 2023Contextual Multilingual Spellchecker for User Queriescited by 2
  6. 2023Multi-teacher Distillation for Multilingual Spelling Correctioncited by 1
  7. 2024Contextual Spelling Correction with Language Model for Low-resource Settingcited by 1
  8. 2024A Comprehensive Approach to Misspelling Correction with BERT and Levenshtein Distancecited by 0
  9. 2026Spelling Correction in Healthcare Query-Answer Systems: Methods, Retrieval Impact, and Empirical Evaluationcited by 0