# Misspelling Oblivious Word Embeddings

arXiv: https://arxiv.org/abs/1905.09755 · HTML: https://mlsanity.com/paper/1905.09755/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 May 2019.

## Built on by (most cited first, 19 of 19)

- [NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation](https://mlsanity.com/paper/2112.02721/) (2021)
- [Improving Zero-Shot Cross-Lingual Transfer Learning via Robust Training](https://mlsanity.com/paper/2104.08645/) (2021)
- [On the Robustness of Intent Classification and Slot Labeling in Goal-oriented Dialog Systems to Real-world Noise](https://mlsanity.com/paper/2104.07149/) (2021)
- [Imputing Out-of-Vocabulary Embeddings with LOVE Makes Language Models Robust with Little Cost](https://mlsanity.com/paper/2203.07860/) (2022)
- [Empirical Error Modeling Improves Robustness of Noisy Neural Sequence Labeling](https://mlsanity.com/paper/2105.11872/) (2021)
- [EDSA-Ensemble: an Event Detection Sentiment Analysis Ensemble Architecture](https://mlsanity.com/paper/2301.12805/) (2023)
- [Data-Driven Mitigation of Adversarial Text Perturbation](https://mlsanity.com/paper/2202.09483/) (2022)
- [Synthetic Data Generation and Joint Learning for Robust Code-Mixed Translation](https://mlsanity.com/paper/2403.16771/) (2024)
- [Towards Entity Alignment in the Open World: An Unsupervised Approach](https://mlsanity.com/paper/2101.10535/) (2021)
- [Analytical Engines With Context-Rich Processing: Towards Efficient Next-Generation Analytics](https://mlsanity.com/paper/2212.07517/) (2022)
- [Computational Models to Study Language Processing in the Human Brain: A Survey](https://mlsanity.com/paper/2403.13368/) (2024)
- [Misspellings in Natural Language Processing: A survey](https://mlsanity.com/paper/2501.16836/) (2025)
- [Robust Embeddings Via Distributions](https://mlsanity.com/paper/2104.08420/) (2021)
- [READIN: A Chinese Multi-Task Benchmark with Realistic and Diverse Input Noises](https://mlsanity.com/paper/2302.07324/) (2023)
- [A Comprehensive Empirical Evaluation of Existing Word Embedding Approaches](https://mlsanity.com/paper/2303.07196/) (2023)
- [Optimizing Context-Enhanced Relational Joins](https://mlsanity.com/paper/2312.01476/) (2023)
- [Learning Robust Named Entity Recognizers From Noisy Data With Retrieval Augmentation](https://mlsanity.com/paper/2407.18562/) (2024)
- [Data Centric Domain Adaptation for Historical Text with OCR Errors](https://mlsanity.com/paper/2107.00927/) (2021)
- [Advancements in eHealth Data Analytics through Natural Language Processing and Deep Learning](https://mlsanity.com/paper/2401.10850/) (2024)

## Abstract (arXiv)

In this paper we present a method to learn word embeddings that are resilient to misspellings. Existing word embeddings have limited applicability to malformed texts, which contain a non-negligible amount of out-of-vocabulary words. We propose a method combining FastText with subwords and a supervised task of learning misspelling patterns. In our method, misspellings of each word are embedded close to their correct variants. We train these embeddings on a new dataset we are releasing publicly. Finally, we experimentally show the advantages of this approach on both intrinsic and extrinsic NLP tasks using public test sets.