# A Unified Benchmark for Evaluating Knowledge Graph Construction Methods and Graph Neural Networks

arXiv: https://arxiv.org/abs/2605.05476 · HTML: https://mlsanity.com/paper/2605.05476/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 32 references that are in the map.
- First posted on arXiv 6 May 2026.

## Builds on (most cited first, 18 of 18)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [Predicting multicellular function through multi-layer tissue networks](https://mlsanity.com/paper/1707.04638/) (2017)
- [MedMentions: A Large Biomedical Corpus Annotated with UMLS Concepts](https://mlsanity.com/paper/1902.09476/) (2019)
- [LLMs4OL: Large Language Models for Ontology Learning](https://mlsanity.com/paper/2307.16648/) (2023)
- [Knowledge Graphs: Opportunities and Challenges](https://mlsanity.com/paper/2303.13948/) (2023)
- [KGGen: Extracting Knowledge Graphs from Plain Text with Language Models](https://mlsanity.com/paper/2502.09956/) (2025)
- [Text2KGBench: A Benchmark for Ontology-Driven Knowledge Graph Generation from Text](https://mlsanity.com/paper/2308.02357/) (2023)
- [Universal Representation Learning of Knowledge Bases by Jointly Embedding Instances and Ontological Concepts](https://mlsanity.com/paper/2103.08115/) (2021)
- [YAGO 4.5: A Large and Clean Knowledge Base with a Rich Taxonomy](https://mlsanity.com/paper/2308.11884/) (2023)
- [QH9: A Quantum Hamiltonian Prediction Benchmark for QM9 Molecules](https://mlsanity.com/paper/2306.09549/) (2023)
- [TransGCN:Coupling Transformation Assumptions with Graph Convolutional Networks for Link Prediction](https://mlsanity.com/paper/1910.00702/) (2019)
- [Automated Knowledge Graph Construction using Large Language Models and Sentence Complexity Modelling](https://mlsanity.com/paper/2509.17289/) (2025)
- [KGrEaT: A Framework to Evaluate Knowledge Graphs via Downstream Tasks](https://mlsanity.com/paper/2308.10537/) (2023)
- [Understanding the Effect of Knowledge Graph Extraction Error on Downstream Graph Analyses: A Case Study on Affiliation Graphs](https://mlsanity.com/paper/2506.12367/) (2025)

## Abstract (arXiv)

Knowledge graphs automatically constructed from text are increasingly used in real-world applications. However, their inherent noise, fragmentation, and semantic inconsistencies significantly affect the performance of Graph Neural Networks (GNNs) on downstream tasks. Assessing their performance and robustness remains difficult, as it is often unclear whether observed results stem from the learning model or from the quality of the constructed graph itself. In this work, we introduce a dual-purpose benchmark designed to jointly evaluate (i) the performance of GNNs on noisy, text-derived graphs and (ii) the effectiveness of graph construction methods on a downstream task. The benchmark is built in the biomedical domain from a single textual corpus and includes two automatically constructed graphs generated using different extraction methods, alongside a high-quality reference graph curated by experts that serves as an upper performance bound. This design enables controlled comparison of construction methods and systematic evaluation of GNN robustness through semi-supervised node classification. We further provide a standardized, reproducible, and extensible evaluation framework, facilitating the integration of new graph extraction methods and learning models.