# Language Model Analysis for Ontology Subsumption Inference

arXiv: https://arxiv.org/abs/2302.06761 · HTML: https://mlsanity.com/paper/2302.06761/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 47 references that are in the map.
- First posted on arXiv 14 Feb 2023.

## Builds on (most cited first, 19 of 19)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [Diffusion-LM Improves Controllable Text Generation](https://mlsanity.com/paper/2205.14217/) (2022)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [Time-Aware Language Models as Temporal Knowledge Bases](https://mlsanity.com/paper/2106.15110/) (2021)
- [Factual Probing Is [MASK]: Learning vs. Learning to Recall](https://mlsanity.com/paper/2104.05240/) (2021)
- [A Review on Language Models as Knowledge Bases](https://mlsanity.com/paper/2204.06031/) (2022)
- [OpenPrompt: An Open-source Framework for Prompt-learning](https://mlsanity.com/paper/2111.01998/) (2021)
- [Can Language Models be Biomedical Knowledge Bases?](https://mlsanity.com/paper/2109.07154/) (2021)
- [Probing Across Time: What Does RoBERTa Know and When?](https://mlsanity.com/paper/2104.07885/) (2021)
- [BERTMap: A BERT-based Ontology Alignment System](https://mlsanity.com/paper/2112.02682/) (2021)
- [Analyzing Commonsense Emergence in Few-shot Knowledge Models](https://mlsanity.com/paper/2101.00297/) (2021)
- [Contextual Semantic Embeddings for Ontology Subsumption Prediction](https://mlsanity.com/paper/2202.09791/) (2022)
- [Ontology-enhanced Prompt-tuning for Few-shot Learning](https://mlsanity.com/paper/2201.11332/) (2022)
- [Are Prompt-based Models Clueless?](https://mlsanity.com/paper/2205.09295/) (2022)

## Built on by (most cited first, 25 of 25)

- [Large Language Models and Knowledge Graphs: Opportunities and Challenges](https://mlsanity.com/paper/2308.06374/) (2023)
- [OLaLa: Ontology Matching with Large Language Models](https://mlsanity.com/paper/2311.03837/) (2023)
- [Language Models as Hierarchy Encoders](https://mlsanity.com/paper/2401.11374/) (2024)
- [Emptying the Ocean with a Spoon: Should We Edit Models?](https://mlsanity.com/paper/2310.11958/) (2023)
- [Agent-OM: Leveraging LLM Agents for Ontology Matching](https://mlsanity.com/paper/2312.00326/) (2023)
- [Editing Conceptual Knowledge for Large Language Models](https://mlsanity.com/paper/2403.06259/) (2024)
- [DeepOnto: A Python Package for Ontology Engineering with Deep Learning](https://mlsanity.com/paper/2307.03067/) (2023)
- [Knowledge Graphs for the Life Sciences: Recent Developments, Challenges and Opportunities](https://mlsanity.com/paper/2309.17255/) (2023)
- [Ontology Enrichment from Texts: A Biomedical Dataset for Concept Discovery and Placement](https://mlsanity.com/paper/2306.14704/) (2023)
- [The LLM Bottleneck: Why Open-Source Vision LLMs Struggle with Hierarchical Visual Recognition](https://mlsanity.com/paper/2505.24840/) (2025)
- [OntoURL: A Benchmark for Evaluating Large Language Models on Symbolic Ontological Understanding, Reasoning and Learning](https://mlsanity.com/paper/2505.11031/) (2025)
- [Ontology Completion with Natural Language Inference and Concept Embeddings: An Analysis](https://mlsanity.com/paper/2403.17216/) (2024)
- [Do LLMs Dream of Ontologies?](https://mlsanity.com/paper/2401.14931/) (2024)
- [A Language Model based Framework for New Concept Placement in Ontologies](https://mlsanity.com/paper/2402.17897/) (2024)
- [Large Language Model for OWL Proofs](https://mlsanity.com/paper/2601.12444/) (2026)
- [Hierarchical Text Classification with LLM-Refined Taxonomies](https://mlsanity.com/paper/2601.18375/) (2026)
- [How Do Large Language Models Learn Concepts During Continual Pre-Training?](https://mlsanity.com/paper/2601.03570/) (2026)
- [From Subsumption to Satisfiability: LLM-Assisted Active Learning for OWL Ontologies](https://mlsanity.com/paper/2604.16672/) (2026)
- [What can knowledge graph alignment gain with Neuro-Symbolic learning approaches?](https://mlsanity.com/paper/2310.07417/) (2023)
- [Transformers in the Service of Description Logic-based Contexts](https://mlsanity.com/paper/2311.08941/) (2023)
- [Ontological Relations from Word Embeddings](https://mlsanity.com/paper/2408.00444/) (2024)
- [Disentangling concept semantics via multilingual averaging in Sparse Autoencoders](https://mlsanity.com/paper/2508.14275/) (2025)
- [Hierarchical Retrieval with Out-Of-Vocabulary Queries: A Case Study on SNOMED CT](https://mlsanity.com/paper/2511.16698/) (2025)
- [NeurOWL: An LLM-Based Neural-symbolic Framework for Incomplete OWL Ontology Reasoning](https://mlsanity.com/paper/2607.15776/) (2026)
- [Ontology-Grounded, Reasoner-Verified Benchmarks for Evaluating LLM Reasoning in Scientific AI](https://mlsanity.com/paper/2610.00682/) (2026)

## Abstract (arXiv)

Investigating whether pre-trained language models (LMs) can function as knowledge bases (KBs) has raised wide research interests recently. However, existing works focus on simple, triple-based, relational KBs, but omit more sophisticated, logic-based, conceptualised KBs such as OWL ontologies. To investigate an LM's knowledge of ontologies, we propose OntoLAMA, a set of inference-based probing tasks and datasets from ontology subsumption axioms involving both atomic and complex concepts. We conduct extensive experiments on ontologies of different domains and scales, and our results demonstrate that LMs encode relatively less background knowledge of Subsumption Inference (SI) than traditional Natural Language Inference (NLI) but can improve on SI significantly when a small number of samples are given. We will open-source our code and datasets.