mlsanityData to 7 Oct 2026

Graph2text or Graph2token: A Perspective of Large Language Models for Graph Learning

arXiv 2501.01124 ↗First posted 2 Jan 2025cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Graphs are data structures used to represent irregular networks and are prevalent in numerous real-world applications. Previous methods directly model graph structures and achieve significant success. However, these methods encounter bottlenecks due to the inherent irregularity of graphs. An innovative solution is converting graphs into textual representations, thereby harnessing the powerful capabilities of Large Language Models (LLMs) to process and comprehend graphs. In this paper, we present a comprehensive review of methodologies for applying LLMs to graphs, termed LLM4graph. The core of LLM4graph lies in transforming graphs into texts for LLMs to understand and analyze. Thus, we propose a novel taxonomy of LLM4graph methods in the view of the transformation. Specifically, existing methods can be divided into two paradigms: Graph2text and Graph2token, which transform graphs into texts or tokens as the input of LLMs, respectively. We point out four challenges during the transformation to systematically present existing methods in a problem-oriented perspective. For practical concerns, we provide a guideline for researchers on selecting appropriate models and LLMs for different graphs and hardware constraints. We also identify five future research directions for LLM4graph.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 94 of its 154 references that are in the map. First posted on arXiv 2 Jan 2025.

Builds on

94 of 154 references.

References · 94

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  6. 2017Graph Attention Networkscited by 8,854
  7. 2017Inductive Representation Learning on Large Graphscited by 6,284
  8. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  9. 2018How Powerful are Graph Neural Networks?cited by 4,355
  10. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  11. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,571
  12. 2014DeepWalk: Online Learning of Social Representationscited by 1,543
38 more
  1. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
  2. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,037
  3. 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 966
  4. 2021E(n) Equivariant Graph Neural Networkscited by 951
  5. 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 878
  6. 2020On the Bottleneck of Graph Neural Networks and its Practical Implicationscited by 624
  7. 2020Knowledge Graphscited by 619
  8. 2019Graph Transformer Networkscited by 520
  9. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  10. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 420
  11. 2020ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Predictioncited by 332
  12. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 271
  13. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 249
  14. 2017Gate-Variants of Gated Recurrent Unit (GRU) Neural Networkscited by 239
  15. 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 209
  16. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 190
  17. 2022Pure Transformers are Powerful Graph Learnerscited by 188
  18. 2019Universal Invariant and Equivariant Graph Neural Networkscited by 182
  19. 2024LLaGA: Large Language and Graph Assistantcited by 177
  20. 2020Graph-Bert: Only Attention is Needed for Learning Graph Representationscited by 176
  21. 2023LLMRec: Large Language Models with Graph Augmentation for Recommendationcited by 171
  22. 2023Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs?cited by 167
  23. 2023On Over-Squashing in Message Passing Neural Networks: The Impact of Width, Depth, and Topologycited by 163
  24. 2022NAGphormer: A Tokenized Graph Transformer for Node Classification in Large Graphscited by 137
  25. 2022A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Languagecited by 129
  26. 2023MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adaptercited by 128
  27. 2019Text Generation from Knowledge Graphs with Graph Transformerscited by 122
  28. 2024Let Your Graph Do the Talking: Encoding Structured Data for LLMscited by 112
  29. 2024Data-efficient Fine-tuning for LLM-based Recommendationcited by 103
  30. 2023Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Surveycited by 92
  31. 2023InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discoverycited by 86
  32. 2019Graph Transformer for Graph-to-Sequence Learningcited by 82
  33. 2023GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Textcited by 82
  34. 2023Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPTcited by 78
  35. 2022Fairness in Graph Mining: A Surveycited by 76
  36. 2019Step-by-Step: Separating Planning from Realization in Neural Data-to-Text Generationcited by 75
  37. 2024OpenGraph: Towards Open Graph Foundation Modelscited by 75
  38. 2021TeraPipe: Token-Level Pipeline Parallelism for Training Large-Scale Language Modelscited by 74

50 most cited of 94 All 94 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 7.

12 papers.

  1. 2025DP-DGAD: A Generalist Dynamic Graph Anomaly Detector with Dynamic Prototypescited by 4
  2. 2026Graph Tokenization for Bridging Graphs and Transformerscited by 3
  3. 2025Fairness in Augmented Graph Learning: A Surveycited by 2
  4. 2025Tackling Distribution Shift in LLM via KILO: Knowledge-Instructed Learning for Continual Adaptationcited by 1
  5. 2026Node Role-Guided LLMs for Dynamic Graph Clusteringcited by 1
  6. 2025Towards Multimodal Graph Large Language Modelcited by 0
  7. 2025Ta-G-T: Subjectivity Capture in Table to Text Generation via RDF Graphscited by 0
  8. 2026Out-of-Distribution Generalization in Graph Foundation Modelscited by 0
  9. 2026A Unified Graph Language Model for Multi-Domain Multi-Task Graph Alignment Instruction Tuningcited by 0
  10. 2026PromptGNN-sim: Deep Fusion and Alignment of GNN and LLMs for Text-Attributed Graph Learningcited by 0
  11. 2026Structurally Speaking: Motif-Oriented Graph Captioning through Bidirectional Graph-Text Translationcited by 0
  12. 2026ZeroGAR: Benchmarking the Adversarial Robustness of Zero-Shot Graph Modelscited by 0