# A Survey of Word Embeddings Evaluation Methods

arXiv: https://arxiv.org/abs/1801.09536 · HTML: https://mlsanity.com/paper/1801.09536/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jan 2018.

## Built on by (most cited first, 26 of 26)

- [PalmTree: Learning an Assembly Language Model for Instruction Embedding](https://mlsanity.com/paper/2103.03809/) (2021)
- [Plurals: A System for Guiding LLMs Via Simulated Social Ensembles](https://mlsanity.com/paper/2409.17213/) (2024)
- [WordBias: An Interactive Visual Tool for Discovering Intersectional Biases Encoded in Word Embeddings](https://mlsanity.com/paper/2103.03598/) (2021)
- [Evaluating Neural Word Embeddings for Sanskrit](https://mlsanity.com/paper/2104.00270/) (2021)
- [PWESuite: Phonetic Word Embeddings and Tasks They Facilitate](https://mlsanity.com/paper/2304.02541/) (2023)
- [Training Cross-Lingual embeddings for Setswana and Sepedi](https://mlsanity.com/paper/2111.06230/) (2021)
- [Zipfian Whitening](https://mlsanity.com/paper/2411.00680/) (2024)
- [Literature review on vulnerability detection using NLP technology](https://mlsanity.com/paper/2104.11230/) (2021)
- [A Survey On Neural Word Embeddings](https://mlsanity.com/paper/2110.01804/) (2021)
- [One System to Rule them All: a Universal Intent Recognition System for Customer Service Chatbots](https://mlsanity.com/paper/2112.08261/) (2021)
- [Assessing the Effectiveness of Syntactic Structure to Learn Code Edit Representations](https://mlsanity.com/paper/2106.06110/) (2021)
- [Ergodic Limits, Relaxations, and Geometric Properties of Random Walk Node Embeddings](https://mlsanity.com/paper/2109.04526/) (2021)
- [Word Embeddings for Automatic Equalization in Audio Mixing](https://mlsanity.com/paper/2202.08898/) (2022)
- [A Comprehensive Empirical Evaluation of Existing Word Embedding Approaches](https://mlsanity.com/paper/2303.07196/) (2023)
- [Towards Fair and Explainable AI using a Human-Centered AI Approach](https://mlsanity.com/paper/2306.07427/) (2023)
- [Linguistically-Informed Neural Architectures for Lexical, Syntactic and Semantic Tasks in Sanskrit](https://mlsanity.com/paper/2308.08807/) (2023)
- [Evaluation of Morphological Embeddings for the Russian Language](https://mlsanity.com/paper/2103.06628/) (2021)
- [Evaluation of Morphological Embeddings for English and Russian Languages](https://mlsanity.com/paper/2103.06884/) (2021)
- [Ground Truths for the Humanities](https://mlsanity.com/paper/2103.12841/) (2021)
- [Evaluating Word Embeddings with Categorical Modularity](https://mlsanity.com/paper/2106.00877/) (2021)
- [An Evaluation Dataset for Legal Word Embedding: A Case Study On Chinese Codex](https://mlsanity.com/paper/2203.15173/) (2022)
- [Bridging Natural Language Processing and Psycholinguistics: computationally grounded semantic similarity datasets for Basque and Spanish](https://mlsanity.com/paper/2304.09616/) (2023)
- [Assessing Linguistic Generalisation in Language Models: A Dataset for Brazilian Portuguese](https://mlsanity.com/paper/2305.14070/) (2023)
- [An Improved Deep Learning Model for Word Embeddings Based Clustering for Large Text Datasets](https://mlsanity.com/paper/2502.16139/) (2025)
- [Intrinsic vs. Extrinsic Evaluation of Czech Sentence Embeddings: Semantic Relevance Doesn't Help with MT Evaluation](https://mlsanity.com/paper/2506.20203/) (2025)
- [BlueGlass: A Framework for Composite AI Safety](https://mlsanity.com/paper/2507.10106/) (2025)

## Abstract (arXiv)

Word embeddings are real-valued word representations able to capture lexical semantics and trained on natural language corpora. Models proposing these representations have gained popularity in the recent years, but the issue of the most adequate evaluation method still remains open. This paper presents an extensive overview of the field of word embeddings evaluation, highlighting main problems and proposing a typology of approaches to evaluation, summarizing 16 intrinsic methods and 12 extrinsic methods. I describe both widely-used and experimental methods, systematize information about evaluation datasets and discuss some key challenges.