mlsanityData to 7 Oct 2026

GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarking

arXiv 2305.15066 ↗First posted 24 May 2023cs.AI cs.CLcited by 209

Not on the map. Why?

Cited by 54 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models~(LLM) like ChatGPT have become indispensable to artificial general intelligence~(AGI), demonstrating excellent performance in various natural language processing tasks. In the real world, graph data is ubiquitous and an essential part of AGI and prevails in domains like social network analysis, bioinformatics and recommender systems. The training corpus of large language models often includes some algorithmic components, which allows them to achieve certain effects on some graph data-related problems. However, there is still little research on their performance on a broader range of graph-structured data. In this study, we conduct an extensive investigation to assess the proficiency of LLMs in comprehending graph data, employing a diverse range of structural and semantic-related tasks. Our analysis encompasses 10 distinct tasks that evaluate the LLMs' capabilities in graph understanding. Through our study, we not only uncover the current limitations of language models in comprehending graph structures and performing associated reasoning tasks but also emphasize the necessity for further advancements and novel approaches to enhance their graph processing capabilities. Our findings contribute valuable insights towards bridging the gap between language models and graph understanding, paving the way for more effective graph mining and knowledge extraction.

Key facts. Cited by 209 papers in the map (data through 7 Oct 2026). Builds on 17 of its 43 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

17 of 43 references.

References · 17

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Graph Attention Networkscited by 8,854
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  7. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  8. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 679
  9. 2020A Survey on Knowledge Graphs: Representation, Acquisition and Applicationscited by 643
  10. 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 339
  11. 2019A Fair Comparison of Graph Neural Networks for Graph Classificationcited by 244
  12. 2020Data Augmentation for Graph Neural Networkscited by 200
5 more
  1. 2017Variational Reasoning for Question Answering with Knowledge Graphcited by 170
  2. 2023Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPTcited by 78
  3. 2021MATE: Multi-view Attention for Table Transformer Efficiencycited by 54
  4. 2022PLOG: Table-to-Logic Pretraining for Logical Table-to-Text Generationcited by 16
  5. 2023Towards Table-to-Text Generation with Pretrained Language Model: A Table Structure Understanding and Text Deliberating Approachcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 29; 2024: 81; 2025: 61; 2026: 38.

209 papers.

Sort:

By area:

  1. 2024Graph Retrieval-Augmented Generation: A Surveycited by 320
  2. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 271
  3. 2023One for All: Towards Training One Graph Model for All Classification Taskscited by 239
  4. 2023Language is All a Graph Needscited by 219
  5. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 190
  6. 2024LLaGA: Large Language and Graph Assistantcited by 177
  7. 2023Talk like a Graph: Encoding Graphs for Large Language Modelscited by 175
  8. 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 170
  9. 2023Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learningcited by 132
  10. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  11. 2024Let Your Graph Do the Talking: Encoding Structured Data for LLMscited by 112
  12. 2023GraphText: Graph Reasoning in Text Spacecited by 110
38 more
  1. 2023GraphLLM: Boosting Graph Reasoning Ability of Large Language Modelcited by 105
  2. 2023Graph Foundation Models: Concepts, Opportunities and Challengescited by 101
  3. 2025TRAIL: Trace Reasoning and Agentic Issue Localizationcited by 98
  4. 2023A Survey of Graph Meets Large Language Model: Progress and Future Directionscited by 84
  5. 2024GFT: Graph Foundation Model with Transferable Tree Vocabularycited by 82
  6. 2023MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Modelscited by 73
  7. 2025Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thoughtcited by 73
  8. 2023Label-free Node Classification on Graphs with Large Language Models (LLMS)cited by 72
  9. 2024A Survey of Large Language Models for Graphscited by 71
  10. 2024Demystifying Chains, Trees, and Graphs of Thoughtscited by 63
  11. 2024LLMs as Zero-shot Graph Learners: Alignment of GNN Representations with LLM Token Embeddingscited by 60
  12. 2024GraphInstruct: Empowering Large Language Models with Graph Understanding and Reasoning Capabilitycited by 57
  13. 2024InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignmentcited by 56
  14. 2023Strategic Behavior of Large Language Models: Game Structure vs. Contextual Framingcited by 49
  15. 2023Making Large Language Models Perform Better in Knowledge Graph Completioncited by 48
  16. 2024Can LLM Graph Reasoning Generalize beyond Pattern Memorization?cited by 46
  17. 2024Paths-over-Graph: Knowledge Graph Empowered Large Language Model Reasoningcited by 44
  18. 2023Amazon-M2: A Multilingual Multi-locale Shopping Session Dataset for Recommendation and Text Generationcited by 42
  19. 2024Can Graph Learning Improve Planning in LLM-based Agents?cited by 42
  20. 2023Thought Propagation: An Analogical Approach to Complex Reasoning with Large Language Modelscited by 37
  21. 2024Graph-enhanced Large Language Models in Asynchronous Plan Reasoningcited by 37
  22. 2023Disentangled Representation Learning with Large Language Models for Text-Attributed Graphscited by 36
  23. 2023Evaluating Large Language Models on Graphs: Performance Insights and Comparative Analysiscited by 35
  24. 2025Capabilities of GPT-5 on Multimodal Medical Reasoningcited by 35
  25. 2024Graph Machine Learning in the Era of Large Language Models (LLMs)cited by 33
  26. 2024Direct Evaluation of Chain-of-Thought in Multi-hop Reasoning with Knowledge Graphscited by 32
  27. 2023Can LLMs Effectively Leverage Graph Structural Information through Prompts, and Why?cited by 31
  28. 2024GITA: Graph to Visual and Textual Integration for Vision-Language Graph Reasoningcited by 31
  29. 2024Exploring Autonomous Agents through the Lens of Large Language Models: A Reviewcited by 30
  30. 2024How Do Large Language Models Understand Graph Patterns? A Benchmark for Graph Pattern Comprehensioncited by 30
  31. 2024Text2Cypher: Bridging Natural Language and Graph Databasescited by 30
  32. 2024Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Treescited by 30
  33. 2024The Heterophilic Graph Learning Handbook: Benchmarks, Models, Theoretical Analysis, Applications and Challengescited by 29
  34. 2023Beyond Text: A Deep Dive into Large Language Models' Ability on Understanding Graph Datacited by 25
  35. 2024Scalable and Accurate Graph Reasoning with LLM-based Multi-Agentscited by 21
  36. 2025GraphICL: Unlocking Graph Learning Potential in LLMs through Structured Prompt Designcited by 21
  37. 2024Graph-oriented Instruction Tuning of Large Language Models for Generic Graph Miningcited by 20
  38. 2023Graph Meets LLMs: Towards Large Graph Modelscited by 19

50 most cited of 209 All 209 (JSON)