OAEI-LLM-T: A TBox Benchmark Dataset for Understanding Large Language Model Hallucinations in Ontology Matching
Not on the map. Why?
Abstract (arXiv)
Hallucinations are often inevitable in downstream tasks using large language models (LLMs). To tackle the substantial challenge of addressing hallucinations for LLM-based ontology matching (OM) systems, we introduce a new benchmark dataset OAEI-LLM-T. The dataset evolves from seven TBox datasets in the Ontology Alignment Evaluation Initiative (OAEI), capturing hallucinations of ten different LLMs performing OM tasks. These OM-specific hallucinations are organised into two primary categories and six sub-categories. We showcase the usefulness of the dataset in constructing an LLM leaderboard for OM tasks and for fine-tuning LLMs used in OM tasks.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 31 of its 41 references that are in the map. First posted on arXiv 25 Mar 2025.
Builds on
31 of 41 references.
References · 31
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
- 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
- 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
- 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
- 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 556
- 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Modelscited by 329
- 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 329
- 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 308
- 2023ToolQA: A Dataset for LLM Question Answering with External Toolscited by 273
- 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 272
- 2022RealTime QA: What's the Answer Right Now?cited by 239
19 more
- 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
- 2023Med-HALT: Medical Domain Hallucination Test for Large Language Modelscited by 156
- 2023Generating Benchmarks for Factuality Evaluation of Language Modelscited by 103
- 2024The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Modelscited by 87
- 2023KoLA: Carefully Benchmarking World Knowledge of Large Language Modelscited by 85
- 2024TofuEval: Evaluating Hallucinations of LLMs on Topic-Focused Dialogue Summarizationcited by 66
- 2023DelucionQA: Detecting Hallucinations in Domain-specific Question Answeringcited by 43
- 2024Confabulation: The Surprising Value of Large Language Model Hallucinationscited by 42
- 2023OLaLa: Ontology Matching with Large Language Modelscited by 35
- 2023Evaluating Hallucinations in Chinese Large Language Modelscited by 33
- 2024HaluEval-Wild: Evaluating Hallucinations of Language Models in the Wildcited by 27
- 2023Exploring Large Language Models for Ontology Alignmentcited by 26
- 2023Agent-OM: Leveraging LLM Agents for Ontology Matchingcited by 26
- 2024LLMs4OM: Matching Ontologies with Large Language Modelscited by 22
- 2024MedHallBench: A New Benchmark for Assessing Hallucination in Medical Large Language Modelscited by 15
- 2023Conversational Ontology Alignment with ChatGPTcited by 11
- 2024Towards Complex Ontology Alignment using Large Language Modelscited by 9
- 2024DAHL: Domain-specific Automated Hallucination Evaluation of Long-Form Text through a Benchmark Dataset in Biomedicinecited by 4
- 2024OAEI-LLM: A Benchmark Dataset for Understanding Large Language Model Hallucinations in Ontology Matchingcited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.