mlsanityData to 14 Oct 2026

Language is All a Graph Needs

arXiv 2308.07134 ↗First posted 14 Aug 2023cs.CL cs.AI cs.IRcited by 220

Not on the map. Why?

Cited by 31 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The emergence of large-scale pre-trained language models has revolutionized various AI research domains. Transformers-based Large Language Models (LLMs) have gradually replaced CNNs and RNNs to unify fields of computer vision and natural language processing. Compared with independent data samples such as images, videos or texts, graphs usually contain rich structural and relational information. Meanwhile, language, especially natural language, being one of the most expressive mediums, excels in describing complex structures. However, existing work on incorporating graph problems into the generative language modeling framework remains very limited. Considering the rising prominence of LLMs, it becomes essential to explore whether LLMs can also replace GNNs as the foundation model for graphs. In this paper, we propose InstructGLM (Instruction-finetuned Graph Language Model) with highly scalable prompts based on natural language instructions. We use natural language to describe multi-scale geometric structure of the graph and then instruction finetune an LLM to perform graph tasks, which enables Generative Graph Learning. Our method surpasses all GNN baselines on ogbn-arxiv, Cora and PubMed datasets, underscoring its effectiveness and sheds light on generative LLMs as new foundation model for graph machine learning. Our code is open-sourced at https://github.com/agiresearch/InstructGLM.

Key facts. Cited by 220 papers in the map (data through 14 Oct 2026). Builds on 84 of its 104 references that are in the map. First posted on arXiv 14 Aug 2023.

Builds on

84 of 104 references.

References · 84

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  7. 2022Training language models to follow instructions with human feedbackcited by 15,971
  8. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  9. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  10. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  11. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  12. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
38 more
  1. 2017Graph Attention Networkscited by 8,923
  2. 2017Inductive Representation Learning on Large Graphscited by 6,320
  3. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  4. 2018How Powerful are Graph Neural Networks?cited by 4,393
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  6. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  9. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  10. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
  11. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  12. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  13. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,089
  14. 2021ViViT: A Video Vision Transformercited by 1,620
  15. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,576
  16. 2019Simplifying Graph Convolutional Networkscited by 1,477
  17. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
  18. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 997
  19. 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 976
  20. 2018Representation Learning on Graphs with Jumping Knowledge Networkscited by 877
  21. 2020Simple and Deep Graph Convolutional Networkscited by 860
  22. 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 826
  23. 2018Hierarchical Graph Representation Learning with Differentiable Poolingcited by 787
  24. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 680
  25. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 668
  26. 2020A Generalization of Transformer Networks to Graphscited by 661
  27. 2019Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological Viewcited by 545
  28. 2020Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classificationcited by 506
  29. 2016Geometric deep learning on graphs and manifolds using mixture model CNNscited by 468
  30. 2019MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixingcited by 443
  31. 2022Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)cited by 402
  32. 2021Beyond Low-frequency Information in Graph Convolutional Networkscited by 362
  33. 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 343
  34. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 274
  35. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 252
  36. 2020DeeperGCN: All You Need to Train Deeper GCNscited by 250
  37. 2020A Note on Over-Smoothing for Graph Neural Networkscited by 249
  38. 2020Graph Random Neural Network for Semi-Supervised Learning on Graphscited by 213

50 most cited of 84 All 84 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 20; 2024: 90; 2025: 65; 2026: 45.

220 papers.

Sort:

By area:

  1. 2024Graph Retrieval-Augmented Generation: A Surveycited by 324
  2. 2024G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answeringcited by 223
  3. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 192
  4. 2024LLaGA: Large Language and Graph Assistantcited by 181
  5. 2023Talk like a Graph: Encoding Graphs for Large Language Modelscited by 176
  6. 2023UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecastingcited by 144
  7. 2024Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphscited by 119
  8. 2024Let Your Graph Do the Talking: Encoding Structured Data for LLMscited by 118
  9. 2023GraphText: Graph Reasoning in Text Spacecited by 111
  10. 2023GraphLLM: Boosting Graph Reasoning Ability of Large Language Modelcited by 107
  11. 2023Graph Foundation Models: Concepts, Opportunities and Challengescited by 103
  12. 2025A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Modelscited by 88
38 more
  1. 2024Position: Graph Foundation Models are Already Herecited by 86
  2. 2023A Survey of Graph Meets Large Language Model: Progress and Future Directionscited by 85
  3. 2024ZeroG: Investigating Cross-dataset Zero-shot Transferability in Graphscited by 85
  4. 2023Large Language Models for Generative Recommendation: A Survey and Visionary Discussionscited by 73
  5. 2023Label-free Node Classification on Graphs with Large Language Models (LLMS)cited by 72
  6. 2023ChatPose: Chatting about 3D Human Posecited by 72
  7. 2024A Survey of Large Language Models for Graphscited by 72
  8. 2024UniGraph: Learning a Unified Cross-Domain Foundation Model for Text-Attributed Graphscited by 66
  9. 2024Demystifying Chains, Trees, and Graphs of Thoughtscited by 63
  10. 2024GraphTranslator: Aligning Graph Model to Large Language Model for Open-ended Taskscited by 63
  11. 2023Compressing LLMs: The Truth is Rarely Pure and Never Simplecited by 62
  12. 2024LLMs as Zero-shot Graph Learners: Alignment of GNN Representations with LLM Token Embeddingscited by 61
  13. 2024GOFA: A Generative One-For-All Model for Joint Graph Language Modelingcited by 57
  14. 2024InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignmentcited by 56
  15. 2024AnyGraph: Graph Foundation Model in the Wildcited by 53
  16. 2023Making Large Language Models Perform Better in Knowledge Graph Completioncited by 48
  17. 2024Can LLM Graph Reasoning Generalize beyond Pattern Memorization?cited by 47
  18. 2024HiGPT: Heterogeneous Graph Language Modelcited by 45
  19. 2023LLM4DyG: Can Large Language Models Solve Spatial-Temporal Problems on Dynamic Graphs?cited by 40
  20. 2024Graph-enhanced Large Language Models in Asynchronous Plan Reasoningcited by 37
  21. 2023Disentangled Representation Learning with Large Language Models for Text-Attributed Graphscited by 36
  22. 2024GLBench: A Comprehensive Benchmark for Graph with Large Language Modelscited by 36
  23. 2024GITA: Graph to Visual and Textual Integration for Vision-Language Graph Reasoningcited by 33
  24. 2024Graph Machine Learning in the Era of Large Language Models (LLMs)cited by 33
  25. 2024LLMs generate structurally realistic social networks but overestimate political homophilycited by 33
  26. 2024GraphWiz: An Instruction-Following Language Model for Graph Problemscited by 32
  27. 2023Can LLMs Effectively Leverage Graph Structural Information through Prompts, and Why?cited by 31
  28. 2024How Do Large Language Models Understand Graph Patterns? A Benchmark for Graph Pattern Comprehensioncited by 30
  29. 2024The Heterophilic Graph Learning Handbook: Benchmarks, Models, Theoretical Analysis, Applications and Challengescited by 29
  30. 2024DTGB: A Comprehensive Benchmark for Dynamic Text-Attributed Graphscited by 27
  31. 2025G-Refer: Graph Retrieval-Augmented Large Language Model for Explainable Recommendationcited by 27
  32. 2023Beyond Text: A Deep Dive into Large Language Models' Ability on Understanding Graph Datacited by 25
  33. 2024FFN-SkipLLM: A Hidden Gem for Autoregressive Decoding with Adaptive Feed Forward Skippingcited by 23
  34. 2025LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Modelscited by 22
  35. 2024Let's Ask GNN: Empowering Large Language Model for Graph In-Context Learningcited by 21
  36. 2025GraphICL: Unlocking Graph Learning Potential in LLMs through Structured Prompt Designcited by 21
  37. 2023Graph Meets LLMs: Towards Large Graph Modelscited by 20
  38. 2024Distilling Large Language Models for Text-Attributed Graph Learningcited by 20

50 most cited of 220 All 220 (JSON)