# A Comprehensive Analysis of Static Word Embeddings for Turkish

arXiv: https://arxiv.org/abs/2405.07778 · HTML: https://mlsanity.com/paper/2405.07778/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 60 references that are in the map.
- First posted on arXiv 13 May 2024.

## Builds on (most cited first, 24 of 24)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [Learning Word Vectors for 157 Languages](https://mlsanity.com/paper/1802.06893/) (2018)
- [Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them](https://mlsanity.com/paper/1903.03862/) (2019)
- [Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes](https://mlsanity.com/paper/1711.08412/) (2017)
- [Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddings](https://mlsanity.com/paper/1904.04047/) (2019)
- [Learned in Translation: Contextualized Word Vectors](https://mlsanity.com/paper/1708.00107/) (2017)
- [Gender-preserving Debiasing for Pre-trained Word Embeddings](https://mlsanity.com/paper/1906.00742/) (2019)
- [Dynamic Coattention Networks For Question Answering](https://mlsanity.com/paper/1611.01604/) (2016)
- [Evaluating Word Embedding Models: Methods and Experimental Results](https://mlsanity.com/paper/1901.09785/) (2019)
- [Are All Good Word Vector Spaces Isomorphic?](https://mlsanity.com/paper/2004.04070/) (2020)
- [A Comparison of Word Embeddings for the Biomedical Natural Language Processing](https://mlsanity.com/paper/1802.00400/) (2018)
- [Problems With Evaluation of Word Embeddings Using Word Similarity Tasks](https://mlsanity.com/paper/1605.02276/) (2016)
- [Towards Better UD Parsing: Deep Contextualized Word Embeddings, Ensemble, and Treebank Concatenation](https://mlsanity.com/paper/1807.03121/) (2018)
- [Obtaining Better Static Word Embeddings Using Contextual Embedding Models](https://mlsanity.com/paper/2106.04302/) (2021)
- [Evaluation of GPT and BERT-based models on identifying protein-protein interactions in biomedical text](https://mlsanity.com/paper/2303.17728/) (2023)
- [Resources for Turkish Dependency Parsing: Introducing the BOUN Treebank and the BoAT Annotation Tool](https://mlsanity.com/paper/2002.10416/) (2020)
- [A Morphology-aware Network for Morphological Disambiguation](https://mlsanity.com/paper/1702.03654/) (2017)
- [Improving Named Entity Recognition by Jointly Learning to Disambiguate Morphological Tags](https://mlsanity.com/paper/1807.06683/) (2018)
- [Comparison of Turkish Word Representations Trained on Different Morphological Forms](https://mlsanity.com/paper/2002.05417/) (2020)

## Built on by (most cited first, 2 of 2)

- [A Comparative Analysis of Static Word Embeddings for Hungarian](https://mlsanity.com/paper/2505.07809/) (2025)
- [TurkEmbed: Turkish Embedding Model on NLI & STS Tasks](https://mlsanity.com/paper/2511.08376/) (2025)

## Abstract (arXiv)

Word embeddings are fixed-length, dense and distributed word representations that are used in natural language processing (NLP) applications. There are basically two types of word embedding models which are non-contextual (static) models and contextual models. The former method generates a single embedding for a word regardless of its context, while the latter method produces distinct embeddings for a word based on the specific contexts in which it appears. There are plenty of works that compare contextual and non-contextual embedding models within their respective groups in different languages. However, the number of studies that compare the models in these two groups with each other is very few and there is no such study in Turkish. This process necessitates converting contextual embeddings into static embeddings. In this paper, we compare and evaluate the performance of several contextual and non-contextual models in both intrinsic and extrinsic evaluation settings for Turkish. We make a fine-grained comparison by analyzing the syntactic and semantic capabilities of the models separately. The results of the analyses provide insights about the suitability of different embedding models in different types of NLP tasks. We also build a Turkish word embedding repository comprising the embedding models used in this work, which may serve as a valuable resource for researchers and practitioners in the field of Turkish NLP. We make the word embeddings, scripts, and evaluation datasets publicly available.