Characterizing Knowledge Graph Tasks in LLM Benchmarks Using Cognitive Complexity Frameworks
Not on the map. Why?
Abstract (arXiv)
Large Language Models (LLMs) are increasingly used for tasks involving Knowledge Graphs (KGs), whose evaluation typically focuses on accuracy and output correctness. We propose a complementary task characterization approach using three complexity frameworks from cognitive psychology. Applying this to the LLM-KG-Bench framework, we highlight value distributions, identify underrepresented demands and motivate richer interpretation and diversity for benchmark evaluation tasks.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 6 of its 11 references that are in the map. First posted on arXiv 17 Sep 2025.
Builds on
6 of 11 references.
References · 6
- 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 668
- 2023LLM-assisted Knowledge Graph Engineering: Experiments with ChatGPTcited by 42
- 2024Assessing SPARQL capabilities of Large Language Modelscited by 14
- 2023Developing a Scalable Benchmark for Assessing Large Language Models in Knowledge Graph Engineeringcited by 13
- 2023Benchmarking the Abilities of Large Language Models for RDF Knowledge Graph Creation and Comprehension: How Well Do LLMs Speak Turtle?cited by 11
- 2025LLM-KG-Bench 3.0: A Compass for SemanticTechnology Capabilities in the Ocean of LLMscited by 3
Built on by
Not cited yet.