# Word Embedding Techniques for Classification of Star Ratings

arXiv: https://arxiv.org/abs/2504.13653 · HTML: https://mlsanity.com/paper/2504.13653/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 36 references that are in the map.
- First posted on arXiv 18 Apr 2025.

## Builds on (most cited first, 12 of 12)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation](https://mlsanity.com/paper/2010.16061/) (2020)
- [Energy Usage Reports: Environmental awareness as part of algorithmic accountability](https://mlsanity.com/paper/1911.08354/) (2019)
- [A Comparison of Word Embeddings for the Biomedical Natural Language Processing](https://mlsanity.com/paper/1802.00400/) (2018)
- [Towards energy-efficient Deep Learning: An overview of energy-efficient approaches along the Deep Learning Lifecycle](https://mlsanity.com/paper/2303.01980/) (2023)
- [Revisiting Word Embeddings in the LLM Era](https://mlsanity.com/paper/2402.11094/) (2024)
- [Word Emdeddings through Hellinger PCA](https://mlsanity.com/paper/1312.5542/) (2013)
- [Visual Word2Vec (vis-w2v): Learning Visually Grounded Word Embeddings Using Abstract Scenes](https://mlsanity.com/paper/1511.07067/) (2015)
- [Language Model Pre-training for Hierarchical Document Representations](https://mlsanity.com/paper/1901.09128/) (2019)
- [Ranking Online Consumer Reviews](https://mlsanity.com/paper/1901.06274/) (2019)
- [A Comparative Study on using Principle Component Analysis with Different Text Classifiers](https://mlsanity.com/paper/1807.03283/) (2018)

## Abstract (arXiv)

Telecom services are at the core of today's societies' everyday needs. The availability of numerous online forums and discussion platforms enables telecom providers to improve their services by exploring the views of their customers to learn about common issues that the customers face. Natural Language Processing (NLP) tools can be used to process the free text collected. One way of working with such data is to represent text as numerical vectors using one of many word embedding models based on neural networks. This research uses a novel dataset of telecom customers' reviews to perform an extensive study showing how different word embedding algorithms can affect the text classification process. Several state-of-the-art word embedding techniques are considered, including BERT, Word2Vec and Doc2Vec, coupled with several classification algorithms. The important issue of feature engineering and dimensionality reduction is addressed and several PCA-based approaches are explored. Moreover, the energy consumption used by the different word embeddings is investigated. The findings show that some word embedding models can lead to consistently better text classifiers in terms of precision, recall and F1-Score. In particular, for the more challenging classification tasks, BERT combined with PCA stood out with the highest performance metrics. Moreover, our proposed PCA approach of combining word vectors using the first principal component shows clear advantages in performance over the traditional approach of taking the average.