# Generalizing Word Embeddings using Bag of Subwords

arXiv: https://arxiv.org/abs/1809.04259 · HTML: https://mlsanity.com/paper/1809.04259/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Sep 2018.

## Built on by (most cited first, 9 of 9)

- [Log-based Anomaly Detection Without Log Parsing](https://mlsanity.com/paper/2108.01955/) (2021)
- [Bridging Subword Gaps in Pretrain-Finetune Paradigm for Natural Language Generation](https://mlsanity.com/paper/2106.06125/) (2021)
- [Imputing Out-of-Vocabulary Embeddings with LOVE Makes Language Models Robust with Little Cost](https://mlsanity.com/paper/2203.07860/) (2022)
- [From Word Vectors to Multimodal Embeddings: Techniques, Applications, and Future Directions For Large Language Models](https://mlsanity.com/paper/2411.05036/) (2024)
- [drsphelps at SemEval-2022 Task 2: Learning idiom representations using BERTRAM](https://mlsanity.com/paper/2204.02821/) (2022)
- [Integrating Approaches to Word Representation](https://mlsanity.com/paper/2109.04876/) (2021)
- [Leveraging knowledge graphs to update scientific word embeddings using latent semantic imputation](https://mlsanity.com/paper/2210.15358/) (2022)
- [Search Intenion Network for Personalized Query Auto-Completion in E-Commerce](https://mlsanity.com/paper/2403.02609/) (2024)
- [Handling Korean Out-of-Vocabulary Words with Phoneme Representation Learning](https://mlsanity.com/paper/2507.04018/) (2025)

## Abstract (arXiv)

We approach the problem of generalizing pre-trained word embeddings beyond fixed-size vocabularies without using additional contextual information. We propose a subword-level word vector generation model that views words as bags of character n-grams. The model is simple, fast to train and provides good vectors for rare or unseen words. Experiments show that our model achieves state-of-the-art performances in English word similarity task and in joint prediction of part-of-speech tag and morphosyntactic attributes in 23 languages, suggesting our model's ability in capturing the relationship between words' textual representations and their embeddings.