# Universal Representation Learning of Knowledge Bases by Jointly Embedding Instances and Ontological Concepts

arXiv: https://arxiv.org/abs/2103.08115 · HTML: https://mlsanity.com/paper/2103.08115/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 45 references that are in the map.
- First posted on arXiv 15 Mar 2021.

## Builds on (most cited first, 18 of 18)

- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [On the Convergence of Adam and Beyond](https://mlsanity.com/paper/1904.09237/) (2019)
- [Exact solutions to the nonlinear dynamics of learning in deep linear neural networks](https://mlsanity.com/paper/1312.6120/) (2013)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [Holographic Embeddings of Knowledge Graphs](https://mlsanity.com/paper/1510.04935/) (2015)
- [Multilingual Knowledge Graph Embeddings for Cross-lingual Knowledge Alignment](https://mlsanity.com/paper/1611.03954/) (2016)
- [Cross-lingual Entity Alignment via Joint Attribute-Preserving Embedding](https://mlsanity.com/paper/1708.05045/) (2017)
- [A Semantic Matching Energy Function for Learning with Multi-relational Data](https://mlsanity.com/paper/1301.3485/) (2013)
- [Learning Symmetric Collaborative Dialogue Agents with Dynamic Knowledge Graph Embeddings](https://mlsanity.com/paper/1704.07130/) (2017)
- [Co-training Embeddings of Knowledge Graphs and Entity Descriptions for Cross-lingual Entity Alignment](https://mlsanity.com/paper/1806.06478/) (2018)
- [Type-Constrained Representation Learning in Knowledge Graphs](https://mlsanity.com/paper/1508.02593/) (2015)
- [Differentiating Concepts and Instances for Knowledge Graph Embedding](https://mlsanity.com/paper/1811.04588/) (2018)
- [Open Question Answering with Weakly Supervised Embedding Models](https://mlsanity.com/paper/1404.4326/) (2014)
- [From Knowledge Graph Embedding to Ontology Embedding? An Analysis of the Compatibility between Vector Space Representations and Rules](https://mlsanity.com/paper/1805.10461/) (2018)
- [Locally Adaptive Translation for Knowledge Graph Embedding](https://mlsanity.com/paper/1512.01370/) (2015)
- [On2Vec: Embedding-based Relation Prediction for Ontology Population](https://mlsanity.com/paper/1809.02382/) (2018)

## Built on by (most cited first, 30 of 37)

- [Knowledge Graph Embedding: A Survey from the Perspective of Representation Spaces](https://mlsanity.com/paper/2211.03536/) (2022)
- [DOM-LM: Learning Generalizable Representations for HTML Documents](https://mlsanity.com/paper/2201.10608/) (2022)
- [Multilingual Knowledge Graph Completion with Self-Supervised Adaptive Graph Alignment](https://mlsanity.com/paper/2203.14987/) (2022)
- [Chain-of-Layer: Iteratively Prompting Large Language Models for Taxonomy Induction from Limited Examples](https://mlsanity.com/paper/2402.07386/) (2024)
- [Dual Box Embeddings for the Description Logic EL++](https://mlsanity.com/paper/2301.11118/) (2023)
- [Neurosymbolic AI for Reasoning over Knowledge Graphs: A Survey](https://mlsanity.com/paper/2302.07200/) (2023)
- [Knowing the No-match: Entity Alignment with Dangling Cases](https://mlsanity.com/paper/2106.02248/) (2021)
- [OntoEA: Ontology-guided Entity Alignment via Joint Knowledge Graph Embedding](https://mlsanity.com/paper/2105.07688/) (2021)
- [CAKE: A Scalable Commonsense-Aware Framework For Multi-View Knowledge Graph Completion](https://mlsanity.com/paper/2202.13785/) (2022)
- [Context-aware Entity Typing in Knowledge Graphs](https://mlsanity.com/paper/2109.07990/) (2021)
- [Dual-Geometric Space Embedding Model for Two-View Knowledge Graphs](https://mlsanity.com/paper/2209.08767/) (2022)
- [SMiLE: Schema-augmented Multi-level Contrastive Learning for Knowledge Graph Link Prediction](https://mlsanity.com/paper/2210.04870/) (2022)
- [Ontology Embedding: A Survey of Methods, Applications and Resources](https://mlsanity.com/paper/2406.10964/) (2024)
- [CORE: A Knowledge Graph Entity Type Prediction Method via Complex Space Regression and Embedding](https://mlsanity.com/paper/2112.10067/) (2021)
- [Metadata-Induced Contrastive Learning for Zero-Shot Multi-Label Text Classification](https://mlsanity.com/paper/2202.05932/) (2022)
- [Learning Representations of Bi-level Knowledge Graphs for Reasoning beyond Link Prediction](https://mlsanity.com/paper/2302.02601/) (2023)
- [Multi-view Inference for Relation Extraction with Uncertain Knowledge](https://mlsanity.com/paper/2104.13579/) (2021)
- [DHGE: Dual-View Hyper-Relational Knowledge Graph Embedding for Link Prediction and Entity Typing](https://mlsanity.com/paper/2207.08562/) (2022)
- [Deep Active Alignment of Knowledge Graph Entities and Schemata](https://mlsanity.com/paper/2304.04389/) (2023)
- [Drug-Target Interaction Prediction with Graph Attention networks](https://mlsanity.com/paper/2107.06099/) (2021)
- [Concept2Box: Joint Geometric Embeddings for Learning Two-View Knowledge Graphs](https://mlsanity.com/paper/2307.01933/) (2023)
- [Semi-supervised Adversarial Learning for Complementary Item Recommendation](https://mlsanity.com/paper/2303.05812/) (2023)
- [Rule Induction in Knowledge Graphs Using Linear Programming](https://mlsanity.com/paper/2110.08245/) (2021)
- [Inductive Meta-path Learning for Schema-complex Heterogeneous Information Networks](https://mlsanity.com/paper/2307.03937/) (2023)
- [Towards Enhancing Relational Rules for Knowledge Graph Link Prediction](https://mlsanity.com/paper/2310.13411/) (2023)
- [Towards Semantically Enriched Embeddings for Knowledge Graph Completion](https://mlsanity.com/paper/2308.00081/) (2023)
- [TAR: Neural Logical Reasoning across TBox and ABox](https://mlsanity.com/paper/2205.14591/) (2022)
- [AsyncET: Asynchronous Learning for Knowledge Graph Entity Typing with Auxiliary Relations](https://mlsanity.com/paper/2308.16055/) (2023)
- [Know2BIO: A Comprehensive Dual-View Benchmark for Evolving Biomedical Knowledge Graphs](https://mlsanity.com/paper/2310.03221/) (2023)
- [Embedding Ontologies via Incorporating Extensional and Intensional Knowledge](https://mlsanity.com/paper/2402.01677/) (2024)

## Abstract (arXiv)

Many large-scale knowledge bases simultaneously represent two views of knowledge graphs (KGs): an ontology view for abstract and commonsense concepts, and an instance view for specific entities that are instantiated from ontological concepts. Existing KG embedding models, however, merely focus on representing one of the two views alone. In this paper, we propose a novel two-view KG embedding model, JOIE, with the goal to produce better knowledge embedding and enable new applications that rely on multi-view knowledge. JOIE employs both cross-view and intra-view modeling that learn on multiple facets of the knowledge base. The cross-view association model is learned to bridge the embeddings of ontological concepts and their corresponding instance-view entities. The intra-view models are trained to capture the structured knowledge of instance and ontology views in separate embedding spaces, with a hierarchy-aware encoding technique enabled for ontologies with hierarchies. We explore multiple representation techniques for the two model components and investigate with nine variants of JOIE. Our model is trained on large-scale knowledge bases that consist of massive instances and their corresponding ontological concepts connected via a (small) set of cross-view links. Experimental results on public datasets show that the best variant of JOIE significantly outperforms previous models on instance-view triple prediction task as well as ontology population on ontologyview KG. In addition, our model successfully extends the use of KG embeddings to entity typing with promising performance.