mlsanityData to 7 Oct 2026

Graph Meets LLMs: Towards Large Graph Models

arXiv 2308.14522 ↗First posted 28 Aug 2023cs.LG cs.AI cs.SIcited by 19

Not on the map. Why?

Abstract (arXiv)

Large models have emerged as the most recent groundbreaking achievements in artificial intelligence, and particularly machine learning. However, when it comes to graphs, large models have not achieved the same level of success as in other fields, such as natural language processing and computer vision. In order to promote applying large models for graphs forward, we present a perspective paper to discuss the challenges and opportunities associated with developing large graph models. First, we discuss the desired characteristics of large graph models. Then, we present detailed discussions from three key perspectives: representation basis, graph data, and graph models. In each category, we provide a brief overview of recent advances and highlight the remaining challenges together with our visions. Finally, we discuss valuable applications of large graph models. We believe this perspective can encourage further investigations into large graph models, ultimately pushing us one step closer towards artificial general intelligence (AGI). We are the first to comprehensively study large graph models, to the best of our knowledge.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 69 of its 114 references that are in the map. First posted on arXiv 28 Aug 2023.

Builds on

69 of 114 references.

References · 69

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2022Training language models to follow instructions with human feedbackcited by 15,712
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  8. 2023Segment Anythingcited by 8,158
  9. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  10. 2020Scaling Laws for Neural Language Modelscited by 6,411
  11. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  12. 2023A Survey of Large Language Modelscited by 2,537
38 more
  1. 2022Emergent Abilities of Large Language Modelscited by 2,483
  2. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  3. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  4. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  5. 2023BloombergGPT: A Large Language Model for Financecited by 923
  6. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  7. 2022A Survey on In-context Learningcited by 717
  8. 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 702
  9. 2020Benchmarking Graph Neural Networkscited by 672
  10. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
  11. 2021Understanding over-squashing and bottlenecks on graphs via curvaturecited by 471
  12. 2023A Survey on Large Language Models for Recommendationcited by 378
  13. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  14. 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 339
  15. 2021OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphscited by 328
  16. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 271
  17. 2023A Survey on Model Compression for Large Language Modelscited by 263
  18. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 249
  19. 2023A Survey on Oversmoothing in Graph Neural Networkscited by 246
  20. 2023FinGPT: Open-Source Financial Large Language Modelscited by 244
  21. 2023Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graphcited by 235
  22. 2017Learning to Represent Programs with Graphscited by 219
  23. 2023Language is All a Graph Needscited by 219
  24. 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 209
  25. 2022Structure-Aware Transformer for Graph Representation Learningcited by 191
  26. 2022Pure Transformers are Powerful Graph Learnerscited by 188
  27. 2023A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detectioncited by 145
  28. 2023Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Surveycited by 140
  29. 2022NAGphormer: A Tokenized Graph Transformer for Node Classification in Large Graphscited by 137
  30. 2023Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learningcited by 132
  31. 2022Transformer for Graphs: An Overview from Architecture Perspectivecited by 107
  32. 2022Out-Of-Distribution Generalization on Graphs: A Surveycited by 97
  33. 2023GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Textcited by 82
  34. 2023Attending to Graph Transformerscited by 81
  35. 2023Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPTcited by 78
  36. 2021A Review on Graph Neural Network Methods in Financial Applicationscited by 77
  37. 2023SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learningcited by 73
  38. 2023MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Modelscited by 73

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 10; 2025: 4; 2026: 3.

19 papers.

Sort:

By area:

  1. 2024G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answeringcited by 217
  2. 2023Graph Foundation Models: Concepts, Opportunities and Challengescited by 101
  3. 2024Position: Graph Foundation Models are Already Herecited by 84
  4. 2024Demystifying Chains, Trees, and Graphs of Thoughtscited by 63
  5. 2024GOFA: A Generative One-For-All Model for Joint Graph Language Modelingcited by 56
  6. 2024KG-RAG: Bridging the Gap Between Knowledge and Creativitycited by 55
  7. 2024Large Language Model-driven Meta-structure Discovery in Heterogeneous Information Networkcited by 16
  8. 2024Exploring the Potential of Large Language Models in Graph Generationcited by 15
  9. 2024CLEAR: Can Language Models Really Understand Causal Graphs?cited by 14
  10. 2025Graph2text or Graph2token: A Perspective of Large Language Models for Graph Learningcited by 12
  11. 2024A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspectivecited by 9
  12. 2025Applications of Large Models in Medicinecited by 6
7 more
  1. 2025Towards Graph Foundation Models: A Transferability Perspectivecited by 5
  2. 2024Edge Graph Intelligence: Reciprocally Empowering Edge Networks with Graph Intelligencecited by 2
  3. 2026Retrieving Minimal and Sufficient Reasoning Subgraphs with Graph Foundation Models for Path-aware GraphRAGcited by 2
  4. 2023Topology Only Pre-Training: Towards Generalised Multi-Domain Graph Modelscited by 1
  5. 2026GraphSeek: Next-Generation Graph Analytics with LLMscited by 1
  6. 2025Adaptive Tokenization: On the Hop-Overpriority Problem in Tokenized Graph Learning Modelscited by 0
  7. 2026HyGRAIL: Cost-Aware and Evidence-Grounded Scientific Hypothesis Discovery over Knowledge Graphscited by 0