# Beyond Word Embeddings: Learning Entity and Concept Representations from Large Scale Knowledge Bases

arXiv: https://arxiv.org/abs/1801.00388 · HTML: https://mlsanity.com/paper/1801.00388/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jan 2018.

## Built on by (most cited first, 6 of 6)

- [Counterfactual Learning on Graphs: A Survey](https://mlsanity.com/paper/2304.01391/) (2023)
- [Entity Concept-enhanced Few-shot Relation Extraction](https://mlsanity.com/paper/2106.02401/) (2021)
- [DOCENT: Learning Self-Supervised Entity Representations from Large Document Collections](https://mlsanity.com/paper/2102.13247/) (2021)
- [Structural Textile Pattern Recognition and Processing Based on Hypergraphs](https://mlsanity.com/paper/2103.11271/) (2021)
- [Leveraging knowledge graphs to update scientific word embeddings using latent semantic imputation](https://mlsanity.com/paper/2210.15358/) (2022)
- [LiCQA : A Lightweight Complex Question Answering System](https://mlsanity.com/paper/2602.22182/) (2026)

## Abstract (arXiv)

Text representations using neural word embeddings have proven effective in many NLP applications. Recent researches adapt the traditional word embedding models to learn vectors of multiword expressions (concepts/entities). However, these methods are limited to textual knowledge bases (e.g., Wikipedia). In this paper, we propose a novel and simple technique for integrating the knowledge about concepts from two large scale knowledge bases of different structure (Wikipedia and Probase) in order to learn concept representations. We adapt the efficient skip-gram model to seamlessly learn from the knowledge in Wikipedia text and Probase concept graph. We evaluate our concept embedding models on two tasks: (1) analogical reasoning, where we achieve a state-of-the-art performance of 91% on semantic analogies, (2) concept categorization, where we achieve a state-of-the-art performance on two benchmark datasets achieving categorization accuracy of 100% on one and 98% on the other. Additionally, we present a case study to evaluate our model on unsupervised argument type identification for neural semantic parsing. We demonstrate the competitive accuracy of our unsupervised method and its ability to better generalize to out of vocabulary entity mentions compared to the tedious and error prone methods which depend on gazetteers and regular expressions.