# Contextual Spelling Correction with Language Model for Low-resource Setting

arXiv: https://arxiv.org/abs/2404.18072 · HTML: https://mlsanity.com/paper/2404.18072/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 32 references that are in the map.
- First posted on arXiv 28 Apr 2024.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [NeuSpell: A Neural Spelling Correction Toolkit](https://mlsanity.com/paper/2010.11085/) (2020)
- [Spelling Correction with Denoising Transformer](https://mlsanity.com/paper/2105.05977/) (2021)
- [uChecker: Masked Pretrained Language Models as Unsupervised Chinese Spelling Checkers](https://mlsanity.com/paper/2209.07068/) (2022)
- [Evaluating GPT-3.5 and GPT-4 on Grammatical Error Correction for Brazilian Portuguese](https://mlsanity.com/paper/2306.15788/) (2023)
- [Unsupervised Context-Sensitive Spelling Correction of English and Dutch Clinical Free-Text with Word and Character N-Gram Embeddings](https://mlsanity.com/paper/1710.07045/) (2017)
- [Beqi: Revitalize the Senegalese Wolof Language with a Robust Spelling Corrector](https://mlsanity.com/paper/2305.08518/) (2023)
- [Does Correction Remain A Problem For Large Language Models?](https://mlsanity.com/paper/2308.01776/) (2023)
- [Vartani Spellcheck -- Automatic Context-Sensitive Spelling Correction of OCR-generated Hindi Text Using BERT and Levenshtein Distance](https://mlsanity.com/paper/2012.07652/) (2020)

## Built on by (most cited first, 1 of 1)

- [Contextual Biasing to Improve Domain-specific Custom Vocabulary Audio Transcription without Explicit Fine-Tuning of Whisper Model](https://mlsanity.com/paper/2410.18363/) (2024)

## Abstract (arXiv)

The task of Spell Correction(SC) in low-resource languages presents a significant challenge due to the availability of only a limited corpus of data and no annotated spelling correction datasets. To tackle these challenges a small-scale word-based transformer LM is trained to provide the SC model with contextual understanding. Further, the probabilistic error rules are extracted from the corpus in an unsupervised way to model the tendency of error happening(error model). Then the combination of LM and error model is used to develop the SC model through the well-known noisy channel framework. The effectiveness of this approach is demonstrated through experiments on the Nepali language where there is access to just an unprocessed corpus of textual data.