# An Ensemble Method to Produce High-Quality Word Embeddings (2016)

arXiv: https://arxiv.org/abs/1604.01692 · HTML: https://mlsanity.com/paper/1604.01692/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Apr 2016.

## Built on by (most cited first, 13 of 13)

- [All Labels Are Not Created Equal: Enhancing Semi-supervision via Label Grouping and Co-training](https://mlsanity.com/paper/2104.05248/) (2021)
- [FacTeR-Check: Semi-automated fact-checking through Semantic Similarity and Natural Language Inference](https://mlsanity.com/paper/2110.14532/) (2021)
- [RetroGAN: A Cyclic Post-Specialization System for Improving Out-of-Knowledge and Rare Word Representations](https://mlsanity.com/paper/2108.12941/) (2021)
- [Seeing the advantage: visually grounding word embeddings to better capture human semantic knowledge](https://mlsanity.com/paper/2202.10292/) (2022)
- [Evaluation of taxonomic and neural embedding methods for calculating semantic similarity](https://mlsanity.com/paper/2209.15197/) (2022)
- [Revisiting the Prepositional-Phrase Attachment Problem Using Explicit Commonsense Knowledge](https://mlsanity.com/paper/2102.00924/) (2021)
- [Contextual Text Embeddings for Twi](https://mlsanity.com/paper/2103.15963/) (2021)
- [TAGLETS: A System for Automatic Semi-Supervised Learning with Auxiliary Data](https://mlsanity.com/paper/2111.04798/) (2021)
- [Zero-Shot Learning in Named-Entity Recognition with External Knowledge](https://mlsanity.com/paper/2111.07734/) (2021)
- [Synonym Detection Using Syntactic Dependency And Neural Embeddings](https://mlsanity.com/paper/2209.15202/) (2022)
- [Improving Arabic Multi-Label Emotion Classification using Stacked Embeddings and Hybrid Loss Function](https://mlsanity.com/paper/2410.03979/) (2024)
- [Like a bilingual baby: The advantage of visually grounding a bilingual language model](https://mlsanity.com/paper/2210.05487/) (2022)
- [Semantic Segmentation of Transparent and Opaque Drinking Glasses with the Help of Zero-shot Learning](https://mlsanity.com/paper/2503.15004/) (2025)

## Abstract (arXiv)

A currently successful approach to computational semantics is to represent words as embeddings in a machine-learned vector space. We present an ensemble method that combines embeddings produced by GloVe (Pennington et al., 2014) and word2vec (Mikolov et al., 2013) with structured knowledge from the semantic networks ConceptNet (Speer and Havasi, 2012) and PPDB (Ganitkevitch et al., 2013), merging their information into a common representation with a large, multilingual vocabulary. The embeddings it produces achieve state-of-the-art performance on many word-similarity evaluations. Its score of ρ = .596 on an evaluation of rare words (Luong et al., 2013) is 16% higher than the previous best known system.