# Embedding Words and Senses Together via Joint Knowledge-Enhanced Training

arXiv: https://arxiv.org/abs/1612.02703 · HTML: https://mlsanity.com/paper/1612.02703/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Dec 2016.

## Built on by (most cited first, 9 of 9)

- [LMMS Reloaded: Transformer-based Sense Embeddings for Disambiguation and Beyond](https://mlsanity.com/paper/2105.12449/) (2021)
- [LU-BZU at SemEval-2021 Task 2: Word2Vec and Lemma2Vec performance in Arabic Word-in-Context disambiguation](https://mlsanity.com/paper/2104.08110/) (2021)
- [Large Scale Substitution-based Word Sense Induction](https://mlsanity.com/paper/2110.07681/) (2021)
- [Multi-sense embeddings through a word sense disambiguation process](https://mlsanity.com/paper/2101.08700/) (2021)
- [Enhanced word embeddings using multi-semantic representation through lexical chains](https://mlsanity.com/paper/2101.09023/) (2021)
- [Improving Word Sense Disambiguation in Neural Machine Translation with Salient Document Context](https://mlsanity.com/paper/2311.15507/) (2023)
- [EDS-MEMBED: Multi-sense embeddings based on enhanced distributional semantic structures via a graph walk over word senses](https://mlsanity.com/paper/2103.00232/) (2021)
- [On the Impact of Knowledge-based Linguistic Annotations in the Quality of Scientific Embeddings](https://mlsanity.com/paper/2104.06200/) (2021)
- [Gyan: An Explainable Neuro-Symbolic Language Model](https://mlsanity.com/paper/2605.04759/) (2026)

## Abstract (arXiv)

Word embeddings are widely used in Natural Language Processing, mainly due to their success in capturing semantic information from massive corpora. However, their creation process does not allow the different meanings of a word to be automatically separated, as it conflates them into a single vector. We address this issue by proposing a new model which learns word and sense embeddings jointly. Our model exploits large corpora and knowledge from semantic networks in order to produce a unified vector space of word and sense embeddings. We evaluate the main features of our approach both qualitatively and quantitatively in a variety of tasks, highlighting the advantages of the proposed method in comparison to state-of-the-art word- and sense-based models.