# Traceable LLM-based validation of statements in knowledge graphs

arXiv: https://arxiv.org/abs/2409.07507 · HTML: https://mlsanity.com/paper/2409.07507/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 25 references that are in the map.
- First posted on arXiv 11 Sep 2024.

## Builds on (most cited first, 8 of 8)

- [MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents](https://mlsanity.com/paper/2404.10774/) (2024)
- [ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT](https://mlsanity.com/paper/2302.10205/) (2023)
- [BioRED: A Rich Biomedical Relation Extraction Dataset](https://mlsanity.com/paper/2204.04263/) (2022)
- [How well do LLMs cite relevant medical references? An evaluation framework and analyses](https://mlsanity.com/paper/2402.02008/) (2024)
- [TeamTat: a collaborative text annotation tool](https://mlsanity.com/paper/2004.11894/) (2020)
- [KG-Hub -- Building and Exchanging Biological Knowledge Graphs](https://mlsanity.com/paper/2302.10800/) (2023)
- [Relation Extraction with Fine-Tuned Large Language Models in Retrieval Augmented Generation Frameworks](https://mlsanity.com/paper/2406.14745/) (2024)
- [Improving Recall of Large Language Models: A Model Collaboration Approach for Relational Triple Extraction](https://mlsanity.com/paper/2404.09593/) (2024)

## Built on by (most cited first, 4 of 4)

- [Enabling LLM Knowledge Analysis via Extensive Materialization](https://mlsanity.com/paper/2411.04920/) (2024)
- [Neurosymbolic AI approach to Attribution in Large Language Models](https://mlsanity.com/paper/2410.03726/) (2024)
- [SIGMUS: Semantic Integration for Knowledge Graphs in Multimodal Urban Spaces](https://mlsanity.com/paper/2509.00287/) (2025)
- [Performance Evaluation of LLMs in Automated RDF Knowledge Graph Generation](https://mlsanity.com/paper/2603.29878/) (2026)

## Abstract (arXiv)

This article presents a method for verifying RDF triples using LLMs, with an emphasis on providing traceable arguments. Because the LLMs cannot currently reliably identify the origin of the information used to construct the response to the user prompt, our approach is to avoid using internal LLM factual knowledge altogether. Instead, verified RDF statements are compared to chunks of external documents retrieved through a web search or Wikipedia. To assess the possible application of this retrieval augmented generation (RAG) workflow on biosciences content, we evaluated 1,719 positive statements from the BioRED dataset and the same number of newly generated negative statements. The resulting precision is 88 %, and recall is 44 %. This indicates that the method requires human oversight. We also evaluated the method on the SNLI dataset, which allowed us to compare our approach with models specifically tuned for the natural language inference task. We demonstrate the method on Wikidata, where a SPARQL query is used to automatically retrieve statements needing verification. Overall, the results suggest that LLMs could be used for large-scale verification of statements in KGs, a task previously unfeasible due to human annotation costs.