# GLBench: A Comprehensive Benchmark for Graph with Large Language Models

arXiv: https://arxiv.org/abs/2407.07457 · HTML: https://mlsanity.com/paper/2407.07457/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 70 references that are in the map.
- First posted on arXiv 10 Jul 2024.

## Builds on (most cited first, 30 of 54)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Deep Graph Infomax](https://mlsanity.com/paper/1809.10341/) (2018)
- [Pitfalls of Graph Neural Network Evaluation](https://mlsanity.com/paper/1811.05868/) (2018)
- [Revisiting Semi-Supervised Learning with Graph Embeddings](https://mlsanity.com/paper/1603.08861/) (2016)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [DeepSeek LLM: Scaling Open-Source Language Models with Longtermism](https://mlsanity.com/paper/2401.02954/) (2024)
- [GraphMAE: Self-Supervised Masked Graph Autoencoders](https://mlsanity.com/paper/2205.10803/) (2022)
- [Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs](https://mlsanity.com/paper/2307.03393/) (2023)
- [Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networks](https://mlsanity.com/paper/2007.02901/) (2020)
- [One for All: Towards Training One Graph Model for All Classification Tasks](https://mlsanity.com/paper/2310.00149/) (2023)
- [GraphGPT: Graph Instruction Tuning for Large Language Models](https://mlsanity.com/paper/2310.13023/) (2023)
- [Language is All a Graph Needs](https://mlsanity.com/paper/2308.07134/) (2023)
- [Large Language Models on Graphs: A Comprehensive Survey](https://mlsanity.com/paper/2312.02783/) (2023)
- [LLaGA: Large Language and Graph Assistant](https://mlsanity.com/paper/2402.08170/) (2024)
- [LLMRec: Large Language Models with Graph Augmentation for Recommendation](https://mlsanity.com/paper/2311.00423/) (2023)
- [Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editing](https://mlsanity.com/paper/2212.10789/) (2022)
- [Learning on Large-scale Text-attributed Graphs via Variational Inference](https://mlsanity.com/paper/2210.14709/) (2022)
- [GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph](https://mlsanity.com/paper/2105.02605/) (2021)

## Built on by (most cited first, 30 of 36)

- [GraphCLIP: Enhancing Transferability in Graph Foundation Models for Text-Attributed Graphs](https://mlsanity.com/paper/2410.10329/) (2024)
- [G-Refer: Graph Retrieval-Augmented Large Language Model for Explainable Recommendation](https://mlsanity.com/paper/2502.12586/) (2025)
- [Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunities](https://mlsanity.com/paper/2506.18019/) (2025)
- [When Do LLMs Help With Node Classification? A Comprehensive Analysis](https://mlsanity.com/paper/2502.00829/) (2025)
- [Graph-based Approaches and Functionalities in Retrieval-Augmented Generation: A Comprehensive Survey](https://mlsanity.com/paper/2504.10499/) (2025)
- [GRAPHGPT-O: Synergistic Multimodal Comprehension and Generation on Graphs](https://mlsanity.com/paper/2502.11925/) (2025)
- [Rewarding Graph Reasoning Process makes LLMs more Generalized Reasoners](https://mlsanity.com/paper/2503.00845/) (2025)
- [Can TabPFN Compete with GNNs for Node Classification via Graph Tabularization?](https://mlsanity.com/paper/2512.08798/) (2025)
- [Learning Posterior Predictive Distributions for Node Classification from Synthetic Graph Priors](https://mlsanity.com/paper/2604.19028/) (2026)
- [GraphThought: Graph Combinatorial Optimization with Thought Generation](https://mlsanity.com/paper/2502.11607/) (2025)
- [Graph-MLLM: Harnessing Multimodal Large Language Models for Multimodal Graph Learning](https://mlsanity.com/paper/2506.10282/) (2025)
- [Model Generalization on Text Attribute Graphs: Principles with Large Language Models](https://mlsanity.com/paper/2502.11836/) (2025)
- [UniGTE: Unified Graph-Text Encoding for Zero-Shot Generalization across Graph Tasks and Domains](https://mlsanity.com/paper/2510.16885/) (2025)
- [Graph-KV: Breaking Sequence via Injecting Structural Biases into Large Language Models](https://mlsanity.com/paper/2506.07334/) (2025)
- [IceBerg: Debiased Self-Training for Class-Imbalanced Node Classification](https://mlsanity.com/paper/2502.06280/) (2025)
- [Are LLM-Enhanced Graph Neural Networks Robust against Poisoning Attacks?](https://mlsanity.com/paper/2603.26105/) (2026)
- [Bridging Large Language Models and Graph Structure Learning Models for Robust Representation Learning](https://mlsanity.com/paper/2410.12096/) (2024)
- [Dynamic Bundling with Large Language Models for Zero-Shot Inference on Text-Attributed Graphs](https://mlsanity.com/paper/2505.17599/) (2025)
- [Learning Noise-Resilient and Transferable Graph-Text Alignment via Dynamic Quality Assessment](https://mlsanity.com/paper/2510.19384/) (2025)
- [S-Path-RAG: Semantic-Aware Shortest-Path Retrieval Augmented Generation for Multi-Hop Knowledge Graph Question Answering](https://mlsanity.com/paper/2603.23512/) (2026)
- [CAMERA: Adapting to Semantic Camouflage in Unsupervised Text-Attributed Graph Fraud Detection](https://mlsanity.com/paper/2605.20032/) (2026)
- [Path-LLM: A Shortest-Path-based LLM Learning for Unified Graph Representation](https://mlsanity.com/paper/2408.05456/) (2024)
- [A Graph Talks, But Who's Listening? Rethinking Evaluations for Graph-Language Models](https://mlsanity.com/paper/2508.20583/) (2025)
- [<SOG_k>: One LLM Token for Explicit Graph Structural Understanding](https://mlsanity.com/paper/2602.01771/) (2026)
- [CITE: A Comprehensive Benchmark for Heterogeneous Text-Attributed Graphs on Catalytic Materials](https://mlsanity.com/paper/2508.15392/) (2025)
- [Grounded by Experience: Generative Healthcare Prediction Augmented with Hierarchical Agentic Retrieval](https://mlsanity.com/paper/2511.13293/) (2025)
- [Beyond One-Size-Fits-All: Adaptive Subgraph Denoising for Zero-Shot Graph Learning with Large Language Models](https://mlsanity.com/paper/2603.02938/) (2026)
- [Learning Hierarchical Knowledge in Text-Rich Networks with Taxonomy-Informed Representation Learning](https://mlsanity.com/paper/2603.08159/) (2026)
- [Struc-EMB: The Potential of Structure-Aware Encoding in Language Embeddings](https://mlsanity.com/paper/2510.08774/) (2025)
- [GraphShaper: Geometry-aware Alignment for Improving Transfer Learning in Text-Attributed Graphs](https://mlsanity.com/paper/2510.12085/) (2025)

## Abstract (arXiv)

The emergence of large language models (LLMs) has revolutionized the way we interact with graphs, leading to a new paradigm called GraphLLM. Despite the rapid development of GraphLLM methods in recent years, the progress and understanding of this field remain unclear due to the lack of a benchmark with consistent experimental protocols. To bridge this gap, we introduce GLBench, the first comprehensive benchmark for evaluating GraphLLM methods in both supervised and zero-shot scenarios. GLBench provides a fair and thorough evaluation of different categories of GraphLLM methods, along with traditional baselines such as graph neural networks. Through extensive experiments on a collection of real-world datasets with consistent data processing and splitting strategies, we have uncovered several key findings. Firstly, GraphLLM methods outperform traditional baselines in supervised settings, with LLM-as-enhancers showing the most robust performance. However, using LLMs as predictors is less effective and often leads to uncontrollable output issues. We also notice that no clear scaling laws exist for current GraphLLM methods. In addition, both structures and semantics are crucial for effective zero-shot transfer, and our proposed simple baseline can even outperform several models tailored for zero-shot scenarios. The data and code of the benchmark can be found at https://github.com/NineAbyss/GLBench.