# On Evaluation of Bangla Word Analogies

arXiv: https://arxiv.org/abs/2304.04613 · HTML: https://mlsanity.com/paper/2304.04613/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 19 references that are in the map.
- First posted on arXiv 10 Apr 2023.

## Builds on (most cited first, 10 of 10)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [Language-agnostic BERT Sentence Embedding](https://mlsanity.com/paper/2007.01852/) (2020)
- [Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond](https://mlsanity.com/paper/1812.10464/) (2018)
- [BanglaBERT: Language Model Pretraining and Benchmarks for Low-Resource Language Understanding Evaluation in Bangla](https://mlsanity.com/paper/2101.00204/) (2021)
- [Not Low-Resource Anymore: Aligner Ensembling, Batch Filtering, and New Datasets for Bengali-English Machine Translation](https://mlsanity.com/paper/2009.09359/) (2020)
- [BNLP: Natural language processing toolkit for Bengali language](https://mlsanity.com/paper/2102.00405/) (2021)
- [BanglaParaphrase: A High-Quality Bangla Paraphrase Dataset](https://mlsanity.com/paper/2210.05109/) (2022)
- [Human and Machine Judgements for Russian Semantic Relatedness](https://mlsanity.com/paper/1708.09702/) (2017)

## Built on by (most cited first, 2 of 2)

- [Revisiting Word Embeddings in the LLM Era](https://mlsanity.com/paper/2402.11094/) (2024)
- [Revisiting Word Embeddings in the LLM Era](https://mlsanity.com/paper/2502.19607/) (2025)

## Abstract (arXiv)

This paper presents a high-quality dataset for evaluating the quality of Bangla word embeddings, which is a fundamental task in the field of Natural Language Processing (NLP). Despite being the 7th most-spoken language in the world, Bangla is a low-resource language and popular NLP models fail to perform well. Developing a reliable evaluation test set for Bangla word embeddings are crucial for benchmarking and guiding future research. We provide a Mikolov-style word analogy evaluation set specifically for Bangla, with a sample size of 16678, as well as a translated and curated version of the Mikolov dataset, which contains 10594 samples for cross-lingual research. Our experiments with different state-of-the-art embedding models reveal that Bangla has its own unique characteristics, and current embeddings for Bangla still struggle to achieve high accuracy on both datasets. We suggest that future research should focus on training models with larger datasets and considering the unique morphological characteristics of Bangla. This study represents the first step towards building a reliable NLP system for the Bangla language1.