mlsanityData to 14 Oct 2026

Efficient Non-parametric Estimation of Multiple Embeddings per Word in Vector Space

arXiv 1504.06654 ↗First posted 24 Apr 2015cs.CL stat.MLcited by 35

Not on the map. Why?

Abstract (arXiv)

There is rising interest in vector-space word embeddings and their use in NLP, especially given recent methods for their fast estimation at very large scale. Nearly all this work, however, assumes a single vector per word type ignoring polysemy and thus jeopardizing their usefulness for downstream tasks. We present an extension to the Skip-gram model that efficiently learns multiple embeddings per word type. It differs from recent related work by jointly performing word sense discrimination and embedding learning, by non-parametrically estimating the number of senses per word type, and by its efficiency and scalability. We present new state-of-the-art results in the word similarity in context task and demonstrate its scalability by training with one machine on a corpus of nearly 1 billion tokens in less than 6 hours.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). First posted on arXiv 24 Apr 2015.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 14; 2022: 8; 2023: 6; 2024: 5; 2025: 2.

35 papers.

Sort:

By area:

  1. 2021Probabilistic Embeddings for Cross-Modal Retrievalcited by 151
  2. 2021Let's Play Mono-Poly: BERT Can Reveal Words' Polysemy Level and Partitionability into Sensescited by 39
  3. 2022Probabilistic Representations for Video Contrastive Learningcited by 34
  4. 2022Sense Embeddings are also Biased--Evaluating Social Biases in Static and Contextualised Sense Embeddingscited by 18
  5. 2021LMMS Reloaded: Transformer-based Sense Embeddings for Disambiguation and Beyondcited by 17
  6. 2024Robust Multimodal Learning via Representation Decouplingcited by 17
  7. 2023Driving Context into Text-to-Text Privatizationcited by 13
  8. 2024RICA2: Rubric-Informed, Calibrated Assessment of Actionscited by 13
  9. 2024Word-specific tonal realizations in Mandarincited by 11
  10. 2023Can Word Sense Distribution Detect Semantic Changes of Words?cited by 10
  11. 2021Multi-sense embeddings through a word sense disambiguation processcited by 7
  12. 2022A Survey on Word Meta-Embedding Learningcited by 7
23 more
  1. 2023Introspective Deep Metric Learningcited by 7
  2. 2021Learning Sense-Specific Static Embeddings using Contextualised Word Embeddings as a Proxycited by 6
  3. 2021Quantum Mathematics in Artificial Intelligencecited by 5
  4. 2021PolyLM: Learning about Polysemy through Language Modelingcited by 5
  5. 2021Theoretical foundations and limits of word embeddings: what types of meaning can they capture?cited by 5
  6. 2022Learning Meta Word Embeddings by Unsupervised Weighted Concatenation of Source Embeddingscited by 5
  7. 2021Towards Multi-Sense Cross-Lingual Alignment of Contextual Embeddingscited by 3
  8. 2021Extending Multi-Sense Word Embedding to Phrases and Sentences for Unsupervised Semantic Applicationscited by 3
  9. 2025Multi-Sense Embeddings for Language Models and Knowledge Distillationcited by 3
  10. 2021A Survey On Neural Word Embeddingscited by 2
  11. 2022Discovering Fine-Grained Semantics in Knowledge Graph Relationscited by 2
  12. 2022CKG: Dynamic Representation Based on Context and Knowledge Graphcited by 2
  13. 2021EDS-MEMBED: Multi-sense embeddings based on enhanced distributional semantic structures via a graph walk over word sensescited by 1
  14. 2021Multi-facet Universal Schemacited by 1
  15. 2023A Comprehensive Empirical Evaluation of Existing Word Embedding Approachescited by 1
  16. 2023Together We Make Sense -- Learning Meta-Sense Embeddings from Pretrained Static Sense Embeddingscited by 1
  17. 2021Sense representations for Portuguese: experiments with sense embeddings and deep neural language modelscited by 0
  18. 2022A Comparison of Approaches for Imbalanced Classification Problems in the Context of Retrieving Relevant Documents for an Analysiscited by 0
  19. 2022Introspective Deep Metric Learning for Image Retrievalcited by 0
  20. 2023Word Sense Induction with Knowledge Distillation from BERTcited by 0
  21. 2024Semantic Cells: Evolutional Process to Acquire Sense Diversity of Itemscited by 0
  22. 2024Analyzing Polysemy Evolution Using Semantic Cellscited by 0
  23. 2025The cell as a token: high-dimensional geometry in language models and cell embeddingscited by 0