mlsanityData to 7 Oct 2026

Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs

arXiv 2307.03393 ↗First posted 7 Jul 2023cs.LG cs.AIcited by 271

Not on the map. Why?

Cited by 25 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Learning on Graphs has attracted immense attention due to its wide real-world applications. The most popular pipeline for learning on graphs with textual node attributes primarily relies on Graph Neural Networks (GNNs), and utilizes shallow text embedding as initial node representations, which has limitations in general knowledge and profound semantic understanding. In recent years, Large Language Models (LLMs) have been proven to possess extensive common knowledge and powerful semantic comprehension abilities that have revolutionized existing workflows to handle text data. In this paper, we aim to explore the potential of LLMs in graph machine learning, especially the node classification task, and investigate two possible pipelines: LLMs-as-Enhancers and LLMs-as-Predictors. The former leverages LLMs to enhance nodes' text attributes with their massive knowledge and then generate predictions through GNNs. The latter attempts to directly employ LLMs as standalone predictors. We conduct comprehensive and systematical studies on these two pipelines under various settings. From comprehensive empirical results, we make original observations and find new insights that open new possibilities and suggest promising directions to leverage LLMs for learning on graphs. Our codes and datasets are available at https://github.com/CurryTang/Graph-LLM.

Key facts. Cited by 271 papers in the map (data through 7 Oct 2026). Builds on 71 of its 108 references that are in the map. First posted on arXiv 7 Jul 2023.

Builds on

71 of 108 references.

References · 71

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  7. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  8. 2017Graph Attention Networkscited by 8,854
  9. 2017Inductive Representation Learning on Large Graphscited by 6,284
  10. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  11. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  12. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
38 more
  1. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  2. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  3. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  4. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  5. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  6. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  7. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  8. 2019Language Models as Knowledge Bases?cited by 1,922
  9. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  10. 2023PaLM 2 Technical Reportcited by 1,154
  11. 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 970
  12. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  13. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
  14. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  15. 2022MTEB: Massive Text Embedding Benchmarkcited by 623
  16. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 548
  17. 2019Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networkscited by 540
  18. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  19. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
  20. 2018Graph Convolutional Networks for Text Classificationcited by 345
  21. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
  22. 2019K-BERT: Enabling Language Representation with Knowledge Graphcited by 303
  23. 2020GPT-GNN: Generative Pre-Training of Graph Neural Networkscited by 302
  24. 2019ERNIE: Enhanced Representation through Knowledge Integrationcited by 301
  25. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 286
  26. 2022Long Range Graph Benchmarkcited by 258
  27. 2022Black-Box Tuning for Language-Model-as-a-Servicecited by 254
  28. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 249
  29. 2019Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networkscited by 241
  30. 2023Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender Systemcited by 241
  31. 2022LinkBERT: Pretraining Language Models with Document Linkscited by 209
  32. 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 209
  33. 2023On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspectivecited by 208
  34. 2023Is ChatGPT a Good Recommender? A Preliminary Studycited by 192
  35. 2023Recommender Systems in the Era of Large Language Models (LLMs)cited by 188
  36. 2021Training Graph Neural Networks with 1000 Layerscited by 169
  37. 2023Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approachcited by 168
  38. 2022Deep Bidirectional Language-Knowledge Graph Pretrainingcited by 156

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 33; 2024: 127; 2025: 68; 2026: 43.

271 papers.

Sort:

By area:

  1. 2024Graph Retrieval-Augmented Generation: A Surveycited by 320
  2. 2023One for All: Towards Training One Graph Model for All Classification Taskscited by 239
  3. 2023GraphGPT: Graph Instruction Tuning for Large Language Modelscited by 223
  4. 2023Language is All a Graph Needscited by 219
  5. 2024G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answeringcited by 217
  6. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 190
  7. 2024Large Language Models for Education: A Survey and Outlookcited by 185
  8. 2024LLaGA: Large Language and Graph Assistantcited by 177
  9. 2023Talk like a Graph: Encoding Graphs for Large Language Modelscited by 175
  10. 2024Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphscited by 116
  11. 2024Let Your Graph Do the Talking: Encoding Structured Data for LLMscited by 112
  12. 2023GraphText: Graph Reasoning in Text Spacecited by 110
38 more
  1. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 109
  2. 2023Graph Foundation Models: Concepts, Opportunities and Challengescited by 101
  3. 2024Large Language Models and Games: A Survey and Roadmapcited by 91
  4. 2023A Survey of Graph Meets Large Language Model: Progress and Future Directionscited by 84
  5. 2024Position: Graph Foundation Models are Already Herecited by 84
  6. 2024ZeroG: Investigating Cross-dataset Zero-shot Transferability in Graphscited by 83
  7. 2023Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queriescited by 82
  8. 2024GRAG: Graph Retrieval-Augmented Generationcited by 77
  9. 2023MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Modelscited by 73
  10. 2023Label-free Node Classification on Graphs with Large Language Models (LLMS)cited by 72
  11. 2024Can GNN be Good Adapter for LLMs?cited by 71
  12. 2024A Survey of Large Language Models for Graphscited by 71
  13. 2024Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligencecited by 65
  14. 2024Demystifying Chains, Trees, and Graphs of Thoughtscited by 63
  15. 2023Graph Prompt Learning: A Comprehensive Survey and Beyondcited by 61
  16. 2024LLMs as Zero-shot Graph Learners: Alignment of GNN Representations with LLM Token Embeddingscited by 60
  17. 2023Compressing LLMs: The Truth is Rarely Pure and Never Simplecited by 59
  18. 2024GraphInstruct: Empowering Large Language Models with Graph Understanding and Reasoning Capabilitycited by 57
  19. 2024GraphArena: Evaluating and Exploring Large Language Models on Graph Computationcited by 48
  20. 2024Position: What Can Large Language Models Tell Us about Time Series Analysiscited by 47
  21. 2024Can LLM Graph Reasoning Generalize beyond Pattern Memorization?cited by 46
  22. 2024HiGPT: Heterogeneous Graph Language Modelcited by 44
  23. 2024Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insightscited by 44
  24. 2024Paths-over-Graph: Knowledge Graph Empowered Large Language Model Reasoningcited by 44
  25. 2023Amazon-M2: A Multilingual Multi-locale Shopping Session Dataset for Recommendation and Text Generationcited by 42
  26. 2024DELL: Generating Reactions and Explanations for LLM-Based Misinformation Detectioncited by 41
  27. 2023LLM4DyG: Can Large Language Models Solve Spatial-Temporal Problems on Dynamic Graphs?cited by 40
  28. 2024A Survey of Mambacited by 39
  29. 2023Thought Propagation: An Analogical Approach to Complex Reasoning with Large Language Modelscited by 37
  30. 2024Graph-enhanced Large Language Models in Asynchronous Plan Reasoningcited by 37
  31. 2023Fast Graph Condensation with Structure-based Neural Tangent Kernelcited by 36
  32. 2023Disentangled Representation Learning with Large Language Models for Text-Attributed Graphscited by 36
  33. 2024GLBench: A Comprehensive Benchmark for Graph with Large Language Modelscited by 36
  34. 2024TokenRec: Learning to Tokenize ID for LLM-based Generative Recommendationcited by 35
  35. 2023Exploring Large Language Models for Human Mobility Prediction under Public Eventscited by 34
  36. 2024Elephants Never Forget: Memorization and Learning of Tabular Data in Large Language Modelscited by 34
  37. 2024GraphEdit: Large Language Models for Graph Structure Learningcited by 33
  38. 2024Graph Machine Learning in the Era of Large Language Models (LLMs)cited by 33

50 most cited of 271 All 271 (JSON)