mlsanityData to 14 Oct 2026

Knowledge Graphs are not Created Equal: Exploring the Properties and Structure of Real KGs

arXiv 2311.06414 ↗First posted 10 Nov 2023cs.LG cs.CLcited by 1

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

Despite the recent popularity of knowledge graph (KG) related tasks and benchmarks such as KG embeddings, link prediction, entity alignment and evaluation of the reasoning abilities of pretrained language models as KGs, the structure and properties of real KGs are not well studied. In this paper, we perform a large scale comparative study of 29 real KG datasets from diverse domains such as the natural sciences, medicine, and NLP to analyze their properties and structural patterns. Based on our findings, we make several recommendations regarding KG-based model development and evaluation. We believe that the rich structural information contained in KGs can benefit the development of better KG models across fields and we hope this study will contribute to breaking the existing data silos between different areas of research (e.g., ML, NLP, AI for sciences).

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 26 of its 50 references that are in the map. First posted on arXiv 10 Nov 2023.

Builds on

26 of 50 references.

References · 26

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
  3. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  4. 2019Language Models as Knowledge Bases?cited by 1,936
  5. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,067
  6. 2014Embedding Entities and Relations for Learning and Inference in Knowledge Basescited by 932
  7. 2021OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphscited by 329
  8. 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 195
  9. 2022A Review on Language Models as Knowledge Basescited by 157
  10. 2020A Benchmarking Study of Embedding-based Entity Alignment for Knowledge Graphscited by 120
  11. 2020Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queriescited by 107
  12. 2023Crawling the Internal Knowledge-Base of Language Modelscited by 85
14 more
  1. 2020PyKEEN 1.0: A Python Library for Training and Evaluating Knowledge Graph Embeddingscited by 77
  2. 2022A Survey of Knowledge Enhanced Pre-trained Language Modelscited by 76
  3. 2020Bringing Light Into the Dark: A Large-scale Evaluation of Knowledge Graph Embedding Models Under a Unified Frameworkcited by 70
  4. 2023Knowledge Graphs: Opportunities and Challengescited by 70
  5. 2016SNAP: A General Purpose Network Analysis and Graph Mining Librarycited by 56
  6. 2023Give Us the Facts: Enhancing Large Language Models with Knowledge Graphs for Fact-aware Language Modelingcited by 56
  7. 2022Multimodal learning with graphscited by 54
  8. 2020CSKG: The CommonSense Knowledge Graphcited by 45
  9. 2022Knowledge Graph Embedding: A Survey from the Perspective of Representation Spacescited by 41
  10. 2020Network Medicine Framework for Identifying Drug Repurposing Opportunities for COVID-19cited by 39
  11. 2021Knowledge Enhanced Pretrained Language Models: A Compreshensive Surveycited by 33
  12. 2019OpenBioLink: A benchmarking framework for large-scale biomedical link predictioncited by 28
  13. 2022A Review of Knowledge Graph Completioncited by 8
  14. 2022KGxBoard: Explainable and Interactive Leaderboard for Evaluation of Knowledge Graph Completion Modelscited by 4

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024The Role of Graph Topology in the Performance of Biomedical Knowledge Graph Completion Modelscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmap
  2. 2024Synergizing Knowledge Graphs with Large Language Models: A Comprehensive Review and Future Prospects ↗
  3. 2026Benchmarking Large Language Models for Knowledge Graph Validation
  4. 2023KG-GPT: A General Framework for Reasoning on Knowledge Graphs Using Large Language Models
  5. 2023Text-To-KG Alignment: Comparing Current Methods on Classification Tasks
  6. 2023Rethinking Language Models as Symbolic Knowledge Graphs
  7. 2024Generating Knowledge Graphs from Large Language Models: A Comparative Study of GPT-4, LLaMA 2, and BERT
  8. 2025KG-LLM-Bench: A Scalable Benchmark for Evaluating LLM Reasoning on Textualized Knowledge Graphs