# Ontology Completion with Natural Language Inference and Concept Embeddings: An Analysis

arXiv: https://arxiv.org/abs/2403.17216 · HTML: https://mlsanity.com/paper/2403.17216/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 56 references that are in the map.
- First posted on arXiv 25 Mar 2024.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [Poincar'e Embeddings for Learning Hierarchical Representations](https://mlsanity.com/paper/1705.08039/) (2017)
- [Hyperbolic Entailment Cones for Learning Hierarchical Embeddings](https://mlsanity.com/paper/1804.01882/) (2018)
- [Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders](https://mlsanity.com/paper/2104.08027/) (2021)
- [Probabilistic Embedding of Knowledge Graphs with Box Lattice Measures](https://mlsanity.com/paper/1805.06627/) (2018)
- [Inferring Concept Hierarchies from Text Corpora via Hyperbolic Embeddings](https://mlsanity.com/paper/1902.00913/) (2019)
- [EL Embeddings: Geometric construction of models for the Description Logic EL ++](https://mlsanity.com/paper/1902.10499/) (2019)
- [Contextual Semantic Embeddings for Ontology Subsumption Prediction](https://mlsanity.com/paper/2202.09791/) (2022)
- [Exploring Large Language Models for Ontology Alignment](https://mlsanity.com/paper/2309.07172/) (2023)
- [OPA2Vec: combining formal and informal content of biomedical ontologies to improve similarity-based prediction](https://mlsanity.com/paper/1804.10922/) (2018)
- [Language Model Analysis for Ontology Subsumption Inference](https://mlsanity.com/paper/2302.06761/) (2023)
- [Constructing Taxonomies from Pretrained Language Models](https://mlsanity.com/paper/2010.12813/) (2020)
- [DeepOnto: A Python Package for Ontology Engineering with Deep Learning](https://mlsanity.com/paper/2307.03067/) (2023)
- [Onto2Vec: joint vector-based representation of biological entities and their ontology-based annotations](https://mlsanity.com/paper/1802.00864/) (2018)
- [MirrorWiC: On Eliciting Word-in-Context Representations from Pretrained Language Models](https://mlsanity.com/paper/2109.09237/) (2021)
- [Efficient Concept Induction for Description Logics](https://mlsanity.com/paper/1812.03243/) (2018)
- [Description Logic EL++ Embeddings with Intersectional Closure](https://mlsanity.com/paper/2202.14018/) (2022)
- [Learning Description Logic Ontologies. Five Approaches. Where Do They Stand?](https://mlsanity.com/paper/2104.01193/) (2021)
- [Modelling Commonsense Properties using Pre-Trained Bi-Encoders](https://mlsanity.com/paper/2210.02771/) (2022)
- [Distilling Semantic Concept Embeddings from Contrastively Fine-Tuned Language Models](https://mlsanity.com/paper/2305.09785/) (2023)
- [Representing Meaning with a Combination of Logical and Distributional Models](https://mlsanity.com/paper/1505.06816/) (2015)
- [Modelling General Properties of Nouns by Selectively Averaging Contextualised Embeddings](https://mlsanity.com/paper/2012.07580/) (2020)

## Built on by (most cited first, 4 of 4)

- [OntoURL: A Benchmark for Evaluating Large Language Models on Symbolic Ontological Understanding, Reasoning and Learning](https://mlsanity.com/paper/2505.11031/) (2025)
- [Language Models as Ontology Encoders](https://mlsanity.com/paper/2507.14334/) (2025)
- [Enhancing Geometric Ontology Embeddings for EL++ with Negative Sampling and Deductive Closure Filtering](https://mlsanity.com/paper/2405.04868/) (2024)
- [Text Classification using Graph Convolutional Networks: A Comprehensive Survey](https://mlsanity.com/paper/2410.09399/) (2024)

## Abstract (arXiv)

We consider the problem of finding plausible knowledge that is missing from a given ontology, as a generalisation of the well-studied taxonomy expansion task. One line of work treats this task as a Natural Language Inference (NLI) problem, thus relying on the knowledge captured by language models to identify the missing knowledge. Another line of work uses concept embeddings to identify what different concepts have in common, taking inspiration from cognitive models for category based induction. These two approaches are intuitively complementary, but their effectiveness has not yet been compared. In this paper, we introduce a benchmark for evaluating ontology completion methods and thoroughly analyse the strengths and weaknesses of both approaches. We find that both approaches are indeed complementary, with hybrid strategies achieving the best overall results. We also find that the task is highly challenging for Large Language Models, even after fine-tuning.