# LLM2KB: Constructing Knowledge Bases using instruction tuned context aware Large Language Models

arXiv: https://arxiv.org/abs/2308.13207 · HTML: https://mlsanity.com/paper/2308.13207/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 29 references that are in the map.
- First posted on arXiv 25 Aug 2023.

## Builds on (most cited first, 19 of 19)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [How Can We Know What Language Models Know?](https://mlsanity.com/paper/1911.12543/) (2019)
- [LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://mlsanity.com/paper/2208.07339/) (2022)
- [8-bit Optimizers via Block-wise Quantization](https://mlsanity.com/paper/2110.02861/) (2021)
- [Orca: Progressive Learning from Complex Explanation Traces of GPT-4](https://mlsanity.com/paper/2306.02707/) (2023)
- [Learning How to Ask: Querying LMs with Mixtures of Soft Prompts](https://mlsanity.com/paper/2104.06599/) (2021)
- [Factual Probing Is [MASK]: Learning vs. Learning to Recall](https://mlsanity.com/paper/2104.05240/) (2021)
- [AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts](https://mlsanity.com/paper/2010.15980/) (2020)
- [Stability](https://mlsanity.com/paper/1310.0150/) (2013)
- [BERTese: Learning to Speak to BERT](https://mlsanity.com/paper/2103.05327/) (2021)
- [Inducing Relational Knowledge from BERT](https://mlsanity.com/paper/1911.12753/) (2019)
- [Prompting as Probing: Using Language Models for Knowledge Base Construction](https://mlsanity.com/paper/2208.11057/) (2022)
- [An Empirical Study on Few-shot Knowledge Probing for Pretrained Language Models](https://mlsanity.com/paper/2109.02772/) (2021)
- [Task-specific Pre-training and Prompt Decomposition for Knowledge Graph Population with Language Models](https://mlsanity.com/paper/2208.12539/) (2022)

## Built on by (most cited first, 4 of 4)

- [UniBind: LLM-Augmented Unified and Balanced Representation Space to Bind Them All](https://mlsanity.com/paper/2403.12532/) (2024)
- [Scaling Knowledge Graph Construction through Synthetic Data Generation and Distillation](https://mlsanity.com/paper/2410.16597/) (2024)
- [KGValidator: A Framework for Automatic Validation of Knowledge Graph Construction](https://mlsanity.com/paper/2404.15923/) (2024)
- [Can an LLM Induce a Graph? Investigating Memory Drift and Context Length](https://mlsanity.com/paper/2510.03611/) (2025)

## Abstract (arXiv)

The advent of Large Language Models (LLM) has revolutionized the field of natural language processing, enabling significant progress in various applications. One key area of interest is the construction of Knowledge Bases (KB) using these powerful models. Knowledge bases serve as repositories of structured information, facilitating information retrieval and inference tasks. Our paper proposes LLM2KB, a system for constructing knowledge bases using large language models, with a focus on the Llama 2 architecture and the Wikipedia dataset. We perform parameter efficient instruction tuning for Llama-2-13b-chat and StableBeluga-13B by training small injection models that have only 0.05 % of the parameters of the base models using the Low Rank Adaptation (LoRA) technique. These injection models have been trained with prompts that are engineered to utilize Wikipedia page contexts of subject entities fetched using a Dense Passage Retrieval (DPR) algorithm, to answer relevant object entities for a given subject entity and relation. Our best performing model achieved an average F1 score of 0.6185 across 21 relations in the LM-KBC challenge held at the ISWC 2023 conference.