# Use of a Structured Knowledge Base Enhances Metadata Curation by Large Language Models

arXiv: https://arxiv.org/abs/2404.05893 · HTML: https://mlsanity.com/paper/2404.05893/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 16 references that are in the map.
- First posted on arXiv 8 Apr 2024.

## Builds on (most cited first, 5 of 5)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [A Benchmark to Understand the Role of Knowledge Graphs on Large Language Model's Accuracy for Question Answering on Enterprise SQL Databases](https://mlsanity.com/paper/2311.07509/) (2023)
- [Aligning Biomedical Metadata with Ontologies Using Clustering and Embeddings](https://mlsanity.com/paper/1903.08206/) (2019)
- [The variable quality of metadata about biological samples used in biomedical experiments](https://mlsanity.com/paper/1808.06907/) (2018)
- [The CEDAR Workbench: An Ontology-Assisted Environment for Authoring Metadata that Describe Scientific Experiments](https://mlsanity.com/paper/1905.06480/) (2019)

## Built on by (most cited first, 2 of 2)

- [From Natural Language to SQL: Review of LLM-based Text-to-SQL Systems](https://mlsanity.com/paper/2410.01066/) (2024)
- [CurateGPT: A flexible language-model assisted biocuration tool](https://mlsanity.com/paper/2411.00046/) (2024)

## Abstract (arXiv)

Metadata play a crucial role in ensuring the findability, accessibility, interoperability, and reusability of datasets. This paper investigates the potential of large language models (LLMs), specifically GPT-4, to improve adherence to metadata standards. We conducted experiments on 200 random data records describing human samples relating to lung cancer from the NCBI BioSample repository, evaluating GPT-4's ability to suggest edits for adherence to metadata standards. We computed the adherence accuracy of field name-field value pairs through a peer review process, and we observed a marginal average improvement in adherence to the standard data dictionary from 79% to 80% (p<0.5). We then prompted GPT-4 with domain information in the form of the textual descriptions of CEDAR templates and recorded a significant improvement to 97% from 79% (p<0.01). These results indicate that, while LLMs may not be able to correct legacy metadata to ensure satisfactory adherence to standards when unaided, they do show promise for use in automated metadata curation when integrated with a structured knowledge base