# Increasing the LLM Accuracy for Question Answering: Ontologies to the Rescue!

arXiv: https://arxiv.org/abs/2405.11706 · HTML: https://mlsanity.com/paper/2405.11706/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 16 references that are in the map.
- First posted on arXiv 20 May 2024.

## Builds on (most cited first, 5 of 5)

- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [Knowledge Graphs](https://mlsanity.com/paper/2003.02320/) (2020)
- [InferFix: End-to-End Program Repair with LLMs](https://mlsanity.com/paper/2303.07263/) (2023)
- [A Survey of Learning-based Automated Program Repair](https://mlsanity.com/paper/2301.03270/) (2023)
- [A Benchmark to Understand the Role of Knowledge Graphs on Large Language Model's Accuracy for Question Answering on Enterprise SQL Databases](https://mlsanity.com/paper/2311.07509/) (2023)

## Built on by (most cited first, 18 of 18)

- [Authenticated Delegation and Authorized AI Agents](https://mlsanity.com/paper/2501.09674/) (2025)
- [Is Long Context All You Need? Leveraging LLM's Extended Context for NL2SQL](https://mlsanity.com/paper/2501.12372/) (2025)
- [Large language models for artificial general intelligence (AGI): A survey of foundational principles and approaches](https://mlsanity.com/paper/2501.03151/) (2025)
- [Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoning](https://mlsanity.com/paper/2506.09853/) (2025)
- [Beyond the Surface: Enhancing LLM-as-a-Judge Alignment with Human via Internal Representations](https://mlsanity.com/paper/2508.03550/) (2025)
- [Improving LLM Reasoning through Interpretable Role-Playing Steering](https://mlsanity.com/paper/2506.07335/) (2025)
- [HuggingGraph: Understanding the Supply Chain of LLM Ecosystem](https://mlsanity.com/paper/2507.14240/) (2025)
- [Exploring Prompt Engineering: A Systematic Review with SWOT Analysis](https://mlsanity.com/paper/2410.12843/) (2024)
- [Mining for Species, Locations, Habitats, and Ecosystems from Scientific Papers in Invasion Biology: A Large-Scale Exploratory Study with Large Language Models](https://mlsanity.com/paper/2501.18287/) (2025)
- [Enhancing Large Language Models through Neuro-Symbolic Integration and Ontological Reasoning](https://mlsanity.com/paper/2504.07640/) (2025)
- [Towards Ontology-Enhanced Representation Learning for Large Language Models](https://mlsanity.com/paper/2405.20527/) (2024)
- [Semantic Layers for Reliable LLM-Powered Data Analytics: A Paired Benchmark of Accuracy and Hallucination Across Three Frontier Models](https://mlsanity.com/paper/2604.25149/) (2026)
- [AraTable: Benchmarking LLMs' Reasoning and Understanding of Arabic Tabular Data](https://mlsanity.com/paper/2507.18442/) (2025)
- [KNOW: A Real-World Ontology for Knowledge Capture with Large Language Models](https://mlsanity.com/paper/2405.19877/) (2024)
- [MaxMind: A Memory Loop Network to Enhance Software Productivity based on Large Language Models](https://mlsanity.com/paper/2408.03841/) (2024)
- [Benchmarking the rationality of AI decision making using the transitivity axiom](https://mlsanity.com/paper/2502.10554/) (2025)
- [Open-DeBias: Toward Mitigating Open-Set Bias in Language Models](https://mlsanity.com/paper/2509.23805/) (2025)
- [Bounded Semantic Planning and Deterministic Compilation for Reliable Enterprise Text-to-SQL](https://mlsanity.com/paper/2608.16663/) (2026)

## Abstract (arXiv)

There is increasing evidence that question-answering (QA) systems with Large Language Models (LLMs), which employ a knowledge graph/semantic representation of an enterprise SQL database (i.e. Text-to-SPARQL), achieve higher accuracy compared to systems that answer questions directly on SQL databases (i.e. Text-to-SQL). Our previous benchmark research showed that by using a knowledge graph, the accuracy improved from 16% to 54%. The question remains: how can we further improve the accuracy and reduce the error rate? Building on the observations of our previous research where the inaccurate LLM-generated SPARQL queries followed incorrect paths, we present an approach that consists of 1) Ontology-based Query Check (OBQC): detects errors by leveraging the ontology of the knowledge graph to check if the LLM-generated SPARQL query matches the semantic of ontology and 2) LLM Repair: use the error explanations with an LLM to repair the SPARQL query. Using the chat with the data benchmark, our primary finding is that our approach increases the overall accuracy to 72% including an additional 8% of "I don't know" unknown results. Thus, the overall error rate is 20%. These results provide further evidence that investing knowledge graphs, namely the ontology, provides higher accuracy for LLM powered question answering systems.