mlsanityData to 7 Oct 2026

Large Language Models on Graphs: A Comprehensive Survey

arXiv 2312.02783 ↗First posted 5 Dec 2023cs.CL cs.LGcited by 190

Not on the map. Why?

Cited by 23 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs), such as GPT4 and LLaMA, are creating significant advancements in natural language processing, due to their strong text encoding/decoding ability and newly found emergent capability (e.g., reasoning). While LLMs are mainly designed to process pure texts, there are many real-world scenarios where text data is associated with rich structure information in the form of graphs (e.g., academic networks, and e-commerce networks) or scenarios where graph data is paired with rich textual information (e.g., molecules with descriptions). Besides, although LLMs have shown their pure text-based reasoning ability, it is underexplored whether such ability can be generalized to graphs (i.e., graph-based reasoning). In this paper, we provide a systematic review of scenarios and techniques related to large language models on graphs. We first summarize potential scenarios of adopting LLMs on graphs into three categories, namely pure graphs, text-attributed graphs, and text-paired graphs. We then discuss detailed techniques for utilizing LLMs on graphs, including LLM as Predictor, LLM as Encoder, and LLM as Aligner, and compare the advantages and disadvantages of different schools of models. Furthermore, we discuss the real-world applications of such methods and summarize open-source codes and benchmark datasets. Finally, we conclude with potential future research directions in this fast-growing field. The related source can be found at https://github.com/PeterGriffinJin/Awesome-Language-Model-on-Graphs.

Key facts. Cited by 190 papers in the map (data through 7 Oct 2026). Builds on 164 of its 263 references that are in the map. First posted on arXiv 5 Dec 2023.

Builds on

164 of 263 references.

References · 164

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2017Proximal Policy Optimization Algorithmscited by 14,441
  7. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  8. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  9. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  10. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  11. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  12. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
38 more
  1. 2017Graph Attention Networkscited by 8,854
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2017Inductive Representation Learning on Large Graphscited by 6,284
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  8. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  9. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  10. 2018How Powerful are Graph Neural Networks?cited by 4,355
  11. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  12. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  13. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  14. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  15. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  16. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  17. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  18. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  19. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  20. 2023Mistral 7Bcited by 3,065
  21. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  22. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  23. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  24. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  25. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  26. 2023A Survey of Large Language Modelscited by 2,537
  27. 2022Emergent Abilities of Large Language Modelscited by 2,483
  28. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  29. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  30. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  31. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  32. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  33. 2020LightGCN: Simplifying and Powering Graph Convolution Network for Recommendationcited by 1,134
  34. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,037
  35. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  36. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  37. 2022Galactica: A Large Language Model for Sciencecited by 736
  38. 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 702

50 most cited of 164 All 164 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 76; 2025: 68; 2026: 46.

190 papers.

Sort:

By area:

  1. 2024Graph Retrieval-Augmented Generation: A Surveycited by 320
  2. 2024G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answeringcited by 217
  3. 2024GNN-RAG: Graph Neural Retrieval for Large Language Model Reasoningcited by 128
  4. 2024Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphscited by 116
  5. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  6. 2024Retrieval-Augmented Generation with Knowledge Graphs for Customer Service Question Answeringcited by 95
  7. 2024Position: Graph Foundation Models are Already Herecited by 84
  8. 2024GRAG: Graph Retrieval-Augmented Generationcited by 77
  9. 2024A Survey of Large Language Models for Graphscited by 71
  10. 2024Understanding Transformer Reasoning Capabilities via Graph Algorithmscited by 64
  11. 2024Demystifying Chains, Trees, and Graphs of Thoughtscited by 63
  12. 2024GraphCLIP: Enhancing Transferability in Graph Foundation Models for Text-Attributed Graphscited by 58
38 more
  1. 2024InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignmentcited by 56
  2. 2024A Comprehensive Survey of Foundation Models in Medicinecited by 56
  3. 2024GOFA: A Generative One-For-All Model for Joint Graph Language Modelingcited by 56
  4. 2024KG-RAG: Bridging the Gap Between Knowledge and Creativitycited by 55
  5. 2024Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insightscited by 44
  6. 2024A Scalable Communication Protocol for Networks of Large Language Modelscited by 43
  7. 2024GLBench: A Comprehensive Benchmark for Graph with Large Language Modelscited by 36
  8. 2024Graph Machine Learning in the Era of Large Language Models (LLMs)cited by 33
  9. 2024LLMs generate structurally realistic social networks but overestimate political homophilycited by 31
  10. 2024The Heterophilic Graph Learning Handbook: Benchmarks, Models, Theoretical Analysis, Applications and Challengescited by 29
  11. 2024Multimodal Large Language Models for Inverse Molecular Design with Retrosynthetic Planningcited by 29
  12. 2024Mosaic of Modalities: A Comprehensive Benchmark for Multimodal Graph Learningcited by 28
  13. 2024Heterogeneous Contrastive Learning for Foundation Models and Beyondcited by 27
  14. 2025Code Graph Model (CGM): A Graph-Integrated Large Language Model for Repository-Level Software Engineering Taskscited by 27
  15. 2024What Does the Bot Say? Opportunities and Risks of Large Language Models in Social Media Bot Detectioncited by 24
  16. 2025Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunitiescited by 23
  17. 2024RAM-EHR: Retrieval Augmentation Meets Clinical Predictions on Electronic Health Recordscited by 21
  18. 2025When Do LLMs Help With Node Classification? A Comprehensive Analysiscited by 21
  19. 2024Surveying the MLLM Landscape: A Meta-Review of Current Surveyscited by 20
  20. 2024Mixture of Parrots: Experts improve memorization more than reasoningcited by 20
  21. 2025Graph-based Approaches and Functionalities in Retrieval-Augmented Generation: A Comprehensive Surveycited by 18
  22. 2024A Survey of Large Language Models on Generative Graph Analytics: Query, Learning, and Applicationscited by 17
  23. 2024HIGHT: Hierarchical Graph Tokenization for Molecule-Language Alignmentcited by 17
  24. 2025GRAPHGPT-O: Synergistic Multimodal Comprehension and Generation on Graphscited by 17
  25. 2024Advancing Graph Representation Learning with Large Language Models: A Comprehensive Survey of Techniquescited by 16
  26. 2024Can Graph Descriptive Order Affect Solving Graph Problems with LLMs?cited by 16
  27. 2024VerilogReader: LLM-Aided Hardware Test Generationcited by 16
  28. 2024InstructG2I: Synthesizing Images from Multimodal Attributed Graphscited by 14
  29. 2024GraCoRe: Benchmarking Graph Comprehension and Complex Reasoning in Large Language Modelscited by 13
  30. 2024Can Graph Neural Networks Learn Language with Extremely Weak Text Supervision?cited by 13
  31. 2025GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuningcited by 13
  32. 2024GraphEval36K: Benchmarking Coding and Reasoning Capabilities of Large Language Models on Graph Datasetscited by 12
  33. 2024GraphTeam: Facilitating Large Language Model-based Graph Analysis via Multi-Agent Collaborationcited by 12
  34. 2025Graph2text or Graph2token: A Perspective of Large Language Models for Graph Learningcited by 12
  35. 2025GraphCheck: Breaking Long-Term Text Barriers with Extracted Knowledge Graph-Powered Fact-Checkingcited by 12
  36. 2025MLaGA: Multimodal Large Language and Graph Assistantcited by 12
  37. 2024GLaM: Fine-Tuning Large Language Models for Domain Knowledge Graph Alignment via Neighborhood Partitioning and Generative Subgraph Encodingcited by 11
  38. 2024iText2KG: Incremental Knowledge Graphs Construction Using Large Language Modelscited by 11

50 most cited of 190 All 190 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Exploring Graph Learning Tasks with Pure LLMs: A Comprehensive Benchmark and Investigation
  2. 2024GUNDAM: Aligning Large Language Models with Graph Understanding
  3. 2026Evaluating LLMs on Large-Scale Graph Property Estimation via Random Walks
  4. 2025Actions Speak Louder than Prompts: A Large-Scale Study of LLMs for Graph Inference
  5. 2024Revisiting the Graph Reasoning Ability of Large Language Models: Case Studies in Translation, Connectivity and Shortest Path
  6. 2026Teaching LLMs to See Graphs: Unifying Text and Structural Reasoning
  7. 2025Rethinking and Benchmarking Large Language Models for Graph Reasoning
  8. 2026Position: How can Graphs Help Large Language Models?