# Clustering Word Embeddings with Self-Organizing Maps. Application on LaRoSeDa -- A Large Romanian Sentiment Data Set

arXiv: https://arxiv.org/abs/2101.04197 · HTML: https://mlsanity.com/paper/2101.04197/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 57 references that are in the map.
- First posted on arXiv 11 Jan 2021.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017)
- [Skip-Thought Vectors](https://mlsanity.com/paper/1506.06726/) (2015)
- [Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms](https://mlsanity.com/paper/1805.09843/) (2018)
- [The birth of Romanian BERT](https://mlsanity.com/paper/2009.08712/) (2020)
- [Automated essay scoring with string kernels and word embeddings](https://mlsanity.com/paper/1804.07954/) (2018)
- [LABR: A Large Scale Arabic Sentiment Analysis Benchmark](https://mlsanity.com/paper/1411.6718/) (2014)
- [MOROCO: The Moldavian and Romanian Dialectal Corpus](https://mlsanity.com/paper/1901.06543/) (2019)
- [Vector of Locally-Aggregated Word Embeddings (VLAWE): A Novel Document-level Representation](https://mlsanity.com/paper/1902.08850/) (2019)
- [From Image to Text Classification: A Novel Approach based on Clustering Word Embeddings](https://mlsanity.com/paper/1707.08098/) (2017)
- [UnibucKernel Reloaded: First Place in Arabic Dialect Identification for the Second Year in a Row](https://mlsanity.com/paper/1805.04876/) (2018)
- [Improving the results of string kernels in sentiment analysis and Arabic dialect identification by adapting them to your test set](https://mlsanity.com/paper/1808.08409/) (2018)

## Built on by (most cited first, 8 of 8)

- [Navigating Dataset Documentations in AI: A Large-Scale Analysis of Dataset Cards on Hugging Face](https://mlsanity.com/paper/2401.13822/) (2024)
- [Distilling the Knowledge of Romanian BERTs Using Multiple Teachers](https://mlsanity.com/paper/2112.12650/) (2021)
- [In-Context Former: Lightning-fast Compressing Context for Large Language Model](https://mlsanity.com/paper/2406.13618/) (2024)
- [GrEmLIn: A Repository of Green Baseline Embeddings for 87 Low-Resource Languages Injected with Multilingual Graph Knowledge](https://mlsanity.com/paper/2409.18193/) (2024)
- [Adversarial Capsule Networks for Romanian Satire Detection and Sentiment Analysis](https://mlsanity.com/paper/2306.07845/) (2023)
- ["Vorbecsti Romanecste?" A Recipe to Train Powerful Romanian LLMs with English Instructions](https://mlsanity.com/paper/2406.18266/) (2024)
- [Small Models, Big Impact: Efficient Corpus and Graph-Based Adaptation of Small Multilingual Language Models for Low-Resource Languages](https://mlsanity.com/paper/2502.10140/) (2025)
- [Enhancing Romanian Offensive Language Detection through Knowledge Distillation, Multi-Task Learning, and Data Augmentation](https://mlsanity.com/paper/2409.20498/) (2024)

## Abstract (arXiv)

Romanian is one of the understudied languages in computational linguistics, with few resources available for the development of natural language processing tools. In this paper, we introduce LaRoSeDa, a Large Romanian Sentiment Data Set, which is composed of 15,000 positive and negative reviews collected from one of the largest Romanian e-commerce platforms. We employ two sentiment classification methods as baselines for our new data set, one based on low-level features (character n-grams) and one based on high-level features (bag-of-word-embeddings generated by clustering word embeddings with k-means). As an additional contribution, we replace the k-means clustering algorithm with self-organizing maps (SOMs), obtaining better results because the generated clusters of word embeddings are closer to the Zipf's law distribution, which is known to govern natural language. We also demonstrate the generalization capacity of using SOMs for the clustering of word embeddings on another recently-introduced Romanian data set, for text categorization by topic.