mlsanityData to 14 Oct 2026

LMSpell: Spell Correction with Pre-Trained Language Models

arXiv 2512.05414 ↗First posted 5 Dec 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Spell correction is still a challenging problem for many languages, especially low-resource languages (LRLs). While pre-trained language models (PLMs) have been employed for spell correction, there has been no proper comparison across PLMs. We present the first empirical study on the effectiveness of the three types of PLMs for spell correction across multiple languages, including low-resource languages. We show that even relatively small PLMs such as the 270M-parameter Gemma 3 and mBART50, when fine-tuned on a dataset of only 5k sentences, can outperform rule-based spell correctors, highlighting a practical pathway for building effective spell correction systems with limited data. We also present a case study with Sinhala to shed light on the plight of spell correction for LRLs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 39 references that are in the map. First posted on arXiv 5 Dec 2025.

Builds on

26 of 39 references.

References · 26

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  4. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  5. 2025Gemma 3 Technical Reportcited by 1,824
  6. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  7. 2024The Faiss librarycited by 376
  8. 2020Multilingual Translation with Extensible Multilingual Pretraining and Finetuningcited by 287
  9. 2020Spelling Error Correction with Soft-Masked BERTcited by 90
  10. 2022Some Languages are More Equal than Others: Probing Deeper into the Linguistic Disparity in the NLP Worldcited by 55
  11. 2023On the (In)Effectiveness of Large Language Models for Chinese Text Correctioncited by 54
  12. 2019Survey on Publicly Available Sinhala Natural Language Processing Tools and Researchcited by 46
14 more
  1. 2020A Survey on Transfer Learning in Natural Language Processingcited by 33
  2. 2020NeuSpell: A Neural Spelling Correction Toolkitcited by 32
  3. 2016Robsut Wrod Reocginiton via semi-Character Recurrent Neural Networkcited by 18
  4. 2023Chinese Spelling Correction as Rephrasing Language Modelcited by 15
  5. 2025SinLlama -- A Large Language Model for Sinhalacited by 9
  6. 2020Sentiment Analysis for Sinhala Language using Deep Learning Techniquescited by 8
  7. 2024A Simple yet Effective Training-free Prompt-free Approach to Chinese Spelling Correction Based on Large Language Modelscited by 7
  8. 2023A Methodology for Generative Spelling Correction via Natural Spelling Errors Emulation across Multiple Domains and Languagescited by 5
  9. 2025Mixture of Small and Large Models for Chinese Spelling Checkcited by 5
  10. 2022BSpell: A CNN-Blended BERT Based Bangla Spell Checkercited by 3
  11. 2023Does Correction Remain A Problem For Large Language Models?cited by 3
  12. 2025KinyaColBERT: A Lexically Grounded Retrieval Model for Low-Resource Retrieval-Augmented Generationcited by 2
  13. 2024A Combination of BERT and Transformer for Vietnamese Spelling Correctioncited by 1
  14. 2024Retrieval Augmented Spelling Correction for E-Commerce Applicationscited by 1

Built on by

Not cited yet.