# How do Scaling Laws Apply to Knowledge Graph Engineering Tasks? The Impact of Model Size on Large Language Model Performance

arXiv: https://arxiv.org/abs/2505.16276 · HTML: https://mlsanity.com/paper/2505.16276/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 28 references that are in the map.
- First posted on arXiv 22 May 2025.

## Builds on (most cited first, 18 of 18)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023)
- [Knowledge Graphs](https://mlsanity.com/paper/2003.02320/) (2020)
- [LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities](https://mlsanity.com/paper/2305.13168/) (2023)
- [Text2KGBench: A Benchmark for Ontology-Driven Knowledge Graph Generation from Text](https://mlsanity.com/paper/2308.02357/) (2023)
- [LLM-assisted Knowledge Graph Engineering: Experiments with ChatGPT](https://mlsanity.com/paper/2307.06917/) (2023)
- [StructuredRAG: JSON Response Formatting with Large Language Models](https://mlsanity.com/paper/2408.11061/) (2024)
- [Assessing SPARQL capabilities of Large Language Models](https://mlsanity.com/paper/2409.05925/) (2024)
- [Developing a Scalable Benchmark for Assessing Large Language Models in Knowledge Graph Engineering](https://mlsanity.com/paper/2308.16622/) (2023)
- [Leveraging LLMs in Scholarly Knowledge Graph Question Answering](https://mlsanity.com/paper/2311.09841/) (2023)
- [Benchmarking the Abilities of Large Language Models for RDF Knowledge Graph Creation and Comprehension: How Well Do LLMs Speak Turtle?](https://mlsanity.com/paper/2309.17122/) (2023)
- [Knowledge Engineering using Large Language Models](https://mlsanity.com/paper/2310.00637/) (2023)
- [LLM-KG-Bench 3.0: A Compass for SemanticTechnology Capabilities in the Ocean of LLMs](https://mlsanity.com/paper/2505.13098/) (2025)

## Built on by (most cited first, 1 of 1)

- [ARUQULA -- An LLM based Text2SPARQL Approach using ReAct and Knowledge Graph Exploration Utilities](https://mlsanity.com/paper/2510.02200/) (2025)

## Abstract (arXiv)

When using Large Language Models (LLMs) to support Knowledge Graph Engineering (KGE), one of the first indications when searching for an appropriate model is its size. According to the scaling laws, larger models typically show higher capabilities. However, in practice, resource costs are also an important factor and thus it makes sense to consider the ratio between model performance and costs. The LLM-KG-Bench framework enables the comparison of LLMs in the context of KGE tasks and assesses their capabilities of understanding and producing KGs and KG queries. Based on a dataset created in an LLM-KG-Bench run covering 26 open state-of-the-art LLMs, we explore the model size scaling laws specific to KGE tasks. In our analyses, we assess how benchmark scores evolve between different model size categories. Additionally, we inspect how the general score development of single models and families of models correlates to their size. Our analyses revealed that, with a few exceptions, the model size scaling laws generally also apply to the selected KGE tasks. However, in some cases, plateau or ceiling effects occurred, i.e., the task performance did not change much between a model and the next larger model. In these cases, smaller models could be considered to achieve high cost-effectiveness. Regarding models of the same family, sometimes larger models performed worse than smaller models of the same family. These effects occurred only locally. Hence it is advisable to additionally test the next smallest and largest model of the same family.