# Words are not Equal: Graded Weighting Model for building Composite Document Vectors

arXiv: https://arxiv.org/abs/1512.03549 · HTML: https://mlsanity.com/paper/1512.03549/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Dec 2015.

## Built on by (most cited first, 2 of 2)

- [Embedding generation for text classification of Brazilian Portuguese user reviews: from bag-of-words to transformers](https://mlsanity.com/paper/2212.00587/) (2022)
- [Unsupervised Contextualized Document Representation](https://mlsanity.com/paper/2109.10509/) (2021)

## Abstract (arXiv)

Despite the success of distributional semantics, composing phrases from word vectors remains an important challenge. Several methods have been tried for benchmark tasks such as sentiment classification, including word vector averaging, matrix-vector approaches based on parsing, and on-the-fly learning of paragraph vectors. Most models usually omit stop words from the composition. Instead of such an yes-no decision, we consider several graded schemes where words are weighted according to their discriminatory relevance with respect to its use in the document (e.g., idf). Some of these methods (particularly tf-idf) are seen to result in a significant improvement in performance over prior state of the art. Further, combining such approaches into an ensemble based on alternate classifiers such as the RNN model, results in an 1.6% performance improvement on the standard IMDB movie review dataset, and a 7.01% improvement on Amazon product reviews. Since these are language free models and can be obtained in an unsupervised manner, they are of interest also for under-resourced languages such as Hindi as well and many more languages. We demonstrate the language free aspects by showing a gain of 12% for two review datasets over earlier results, and also release a new larger dataset for future testing (Singh,2015).