mlsanityData to 7 Oct 2026

Talk like a Graph: Encoding Graphs for Large Language Models

arXiv 2310.04560 ↗First posted 6 Oct 2023cs.LGcited by 175

Not on the map. Why?

Cited by 49 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Graphs are a powerful tool for representing and analyzing complex relationships in real-world applications such as social networks, recommender systems, and computational finance. Reasoning on graphs is essential for drawing inferences about the relationships between entities in a complex system, and to identify hidden patterns and trends. Despite the remarkable progress in automated reasoning with natural text, reasoning on graphs with large language models (LLMs) remains an understudied problem. In this work, we perform the first comprehensive study of encoding graph-structured data as text for consumption by LLMs. We show that LLM performance on graph reasoning tasks varies on three fundamental levels: (1) the graph encoding method, (2) the nature of the graph task itself, and (3) interestingly, the very structure of the graph considered. These novel results provide valuable insight on strategies for encoding graphs as text. Using these insights we illustrate how the correct choice of encoders can boost performance on graph reasoning tasks inside LLMs by 4.8% to 61.8%, depending on the task.

Key facts. Cited by 175 papers in the map (data through 7 Oct 2026). Builds on 42 of its 52 references that are in the map. First posted on arXiv 6 Oct 2023.

Builds on

42 of 52 references.

References · 42

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  7. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  10. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  11. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  12. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
30 more
  1. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  2. 2023A Survey of Large Language Modelscited by 2,537
  3. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  4. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  5. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  6. 2023PaLM 2 Technical Reportcited by 1,154
  7. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  8. 2023Large Language Models as Optimizerscited by 830
  9. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  10. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
  11. 2020A Generalization of Transformer Networks to Graphscited by 653
  12. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
  13. 2022Towards Reasoning in Large Language Models: A Surveycited by 593
  14. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554
  15. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 271
  16. 2020Transformers as Soft Reasoners over Languagecited by 265
  17. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 249
  18. 2023Language is All a Graph Needscited by 219
  19. 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 209
  20. 2020Graph-Bert: Only Attention is Needed for Learning Graph Representationscited by 176
  21. 2022Deep Bidirectional Language-Knowledge Graph Pretrainingcited by 156
  22. 2021Generate & Rank: A Multi-task Framework for Math Word Problemscited by 82
  23. 2023Attending to Graph Transformerscited by 81
  24. 2022LAMBADA: Backward Chaining for Automated Reasoning in Natural Languagecited by 66
  25. 2022GraphWorld: Fake Graphs Bring Real Insights for GNNscited by 56
  26. 2023Patton: Language Model Pretraining on Text-Rich Networkscited by 49
  27. 2022A Decade of Knowledge Graphs in Natural Language Processing: A Surveycited by 40
  28. 2023Exploring the MIT Mathematics and EECS Curriculum Using Large Language Modelscited by 10
  29. 2023Examining the Effects of Degree Distribution and Homophily in Graph Learning Modelscited by 5
  30. 2022Hidden Schema Networkscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 67; 2025: 55; 2026: 47.

175 papers.

Sort:

By area:

  1. 2024G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answeringcited by 217
  2. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 190
  3. 2024LLaGA: Large Language and Graph Assistantcited by 177
  4. 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 170
  5. 2024Let Your Graph Do the Talking: Encoding Structured Data for LLMscited by 112
  6. 2023GeomVerse: A Systematic Evaluation of Large Models for Geometric Reasoningcited by 107
  7. 2023A Survey of Graph Meets Large Language Model: Progress and Future Directionscited by 84
  8. 2024Position: Graph Foundation Models are Already Herecited by 84
  9. 2024Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoningcited by 83
  10. 2024GFT: Graph Foundation Model with Transferable Tree Vocabularycited by 82
  11. 2024GRAG: Graph Retrieval-Augmented Generationcited by 77
  12. 2024OpenGraph: Towards Open Graph Foundation Modelscited by 75
38 more
  1. 2025Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thoughtcited by 73
  2. 2024Understanding Transformer Reasoning Capabilities via Graph Algorithmscited by 64
  3. 2024Demystifying Chains, Trees, and Graphs of Thoughtscited by 63
  4. 2023Graph Prompt Learning: A Comprehensive Survey and Beyondcited by 61
  5. 2024GOFA: A Generative One-For-All Model for Joint Graph Language Modelingcited by 56
  6. 2024GraphArena: Evaluating and Exploring Large Language Models on Graph Computationcited by 48
  7. 2024HiGPT: Heterogeneous Graph Language Modelcited by 44
  8. 2024Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insightscited by 44
  9. 2024Graph-enhanced Large Language Models in Asynchronous Plan Reasoningcited by 37
  10. 2024Fully-inductive Node Classification on Arbitrary Graphscited by 36
  11. 2024How Far Can Transformers Reason? The Globality Barrier and Inductive Scratchpadcited by 35
  12. 2024Graph Machine Learning in the Era of Large Language Models (LLMs)cited by 33
  13. 2024GraphWiz: An Instruction-Following Language Model for Graph Problemscited by 32
  14. 2025Relational Transformer: Toward Zero-Shot Foundation Models for Relational Datacited by 32
  15. 2026AIRS-Bench: a Suite of Tasks for Frontier AI Research Science Agentscited by 32
  16. 2024GITA: Graph to Visual and Textual Integration for Vision-Language Graph Reasoningcited by 31
  17. 2024How Do Large Language Models Understand Graph Patterns? A Benchmark for Graph Pattern Comprehensioncited by 30
  18. 2024Do Vision-Language Models Really Understand Visual Language?cited by 29
  19. 2024Network Formation and Dynamics Among Multi-LLMscited by 28
  20. 2024ReMI: A Dataset for Reasoning with Multiple Imagescited by 27
  21. 2025LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Modelscited by 21
  22. 2024Mixture of Parrots: Experts improve memorization more than reasoningcited by 20
  23. 2024Learning Graph Quantized Tokenizerscited by 19
  24. 2024GraphInsight: Unlocking Insights in Large Language Models for Graph Structure Understandingcited by 18
  25. 2025Graph-based Approaches and Functionalities in Retrieval-Augmented Generation: A Comprehensive Surveycited by 18
  26. 2024A Survey of Large Language Models on Generative Graph Analytics: Query, Learning, and Applicationscited by 17
  27. 2024HIGHT: Hierarchical Graph Tokenization for Molecule-Language Alignmentcited by 17
  28. 2025RTL++: Graph-enhanced LLM for RTL Code Generationcited by 16
  29. 2025G1: Teaching LLMs to Reason on Graphs with Reinforcement Learningcited by 14
  30. 2024Large Language Model Meets Graph Neural Network in Knowledge Distillationcited by 13
  31. 2024StrucText-Eval: Evaluating Large Language Model's Reasoning Ability in Structure-Rich Textcited by 13
  32. 2024Revisiting Random Walks for Learning on Graphscited by 13
  33. 2024GraphEval36K: Benchmarking Coding and Reasoning Capabilities of Large Language Models on Graph Datasetscited by 12
  34. 2024GraphTeam: Facilitating Large Language Model-based Graph Analysis via Multi-Agent Collaborationcited by 12
  35. 2025Flatten Graphs as Sequences: Transformers are Scalable Graph Generatorscited by 12
  36. 2024Beyond Graphs: Can Large Language Models Comprehend Hypergraphs?cited by 11
  37. 2025Depth-Width tradeoffs in Algorithmic Reasoning of Graph Tasks with Transformerscited by 11
  38. 2024Revisiting the Graph Reasoning Ability of Large Language Models: Case Studies in Translation, Connectivity and Shortest Pathcited by 10

50 most cited of 175 All 175 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024GraphInstruct: Empowering Large Language Models with Graph Understanding and Reasoning Capability
  2. 2026Are Large Language Models Suitable for Graph Computation? Progress and Prospects
  3. 2024Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs
  4. 2023Evaluating Large Language Models on Graphs: Performance Insights and Comparative Analysis
  5. 2025Rethinking and Benchmarking Large Language Models for Graph Reasoning
  6. 2025Exploring Graph Learning Tasks with Pure LLMs: A Comprehensive Benchmark and Investigation
  7. 2023GraphLLM: Boosting Graph Reasoning Ability of Large Language Model
  8. 2023GraphText: Graph Reasoning in Text Space