mlsanityData to 14 Oct 2026

Do General NLP Embeddings Capture Ontological Reasoning?

arXiv 2609.00177 ↗First posted 31 Aug 2026cs.CL cs.AIcited by 0

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

General-purpose NLP embedding models perform well on linguistic tasks, but their ability to capture symbolic ontological structure remains unclear. We introduce AVA, a systematic framework for evaluating whether embeddings distinguish logic-sensitive relational semantics in ontologies and knowledge graphs. AVA comprises 171,007 contrastive triplets derived from 163 heterogeneous ontologies using hierarchy inversion, relation substitution, and disjointness injection. Each triplet contains an ontology statement, a semantically equivalent paraphrase, and a logic-sensitive hard negative with contradictory relational meaning. We evaluate more than 25 state-of-the-art embedding models and find substantial limitations: the best model achieves only 0.739 triplet accuracy, while hard negative accuracy falls to 0.135. Fine-tuning improves discrimination by a large margin but transfers poorly to downstream Semantic Web tasks, including taxonomy discovery and ontology alignment. Further analysis suggests that improvements stem partly from perturbation-specific pattern recognition rather than robust ontological understanding. These findings reveal a persistent gap between linguistic representation learning and ontology-level discrimination, challenging the assumption that strong NLP benchmark performance translates to Semantic Web competence.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 10 of its 40 references that are in the map. First posted on arXiv 31 Aug 2026.

Builds on

10 of 40 references.

References · 10

  1. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  2. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  3. 2019RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Spacecited by 898
  4. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 723
  5. 2019KG-BERT: BERT for Knowledge Graph Completioncited by 324
  6. 2025LLM-empowered knowledge graph construction: A surveycited by 21
  7. 2025Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaningcited by 10
  8. 2026OntoLearner: A Modular Python Library for Ontology Learning with Large Language Modelscited by 3
  9. 2025Towards Efficient LLM-aware Heterogeneous Graph Learningcited by 2
  10. 2025Advances and Challenges in Semantic Textual Similarity: A Comprehensive Surveycited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Ontology Embedding: A Survey of Methods, Applications and Resources
  2. 2025OntoURL: A Benchmark for Evaluating Large Language Models on Symbolic Ontological Understanding, Reasoning and Learning
  3. 2023Dual Box Embeddings for the Description Logic EL++
  4. 2026LLM-Assisted Discovery of Typed Semantic Links for Ontology Network Construction
  5. 2026CORE: Comprehensive Ontological Relation Evaluation for Large Language Models
  6. 2025OntoAligner Meets Knowledge Graph Embedding Aligners
  7. 2026NeurOWL: An LLM-Based Neural-symbolic Framework for Incomplete OWL Ontology Reasoning
  8. 2025Disentangling concept semantics via multilingual averaging in Sparse Autoencoders