mlsanityData to 7 Oct 2026

Let Your Graph Do the Talking: Encoding Structured Data for LLMs

arXiv 2402.05862 ↗First posted 8 Feb 2024cs.LG cs.AI cs.SIcited by 112

Not on the map. Why?

Cited by 32 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

How can we best encode structured data into sequential form for use in large language models (LLMs)? In this work, we introduce a parameter-efficient method to explicitly represent structured data for LLMs. Our method, GraphToken, learns an encoding function to extend prompts with explicit structured information. Unlike other work which focuses on limited domains (e.g. knowledge graph representation), our work is the first effort focused on the general encoding of structured data to be used for various reasoning tasks. We show that explicitly representing the graph structure allows significant improvements to graph reasoning tasks. Specifically, we see across the board improvements - up to 73% points - on node, edge and, graph-level tasks from the GraphQA benchmark.

Key facts. Cited by 112 papers in the map (data through 7 Oct 2026). Builds on 48 of its 61 references that are in the map. First posted on arXiv 8 Feb 2024.

Builds on

48 of 61 references.

References · 48

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  7. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  8. 2018How Powerful are Graph Neural Networks?cited by 4,355
  9. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  10. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  11. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  12. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
36 more
  1. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  2. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  3. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  4. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  5. 2023A Survey of Large Language Modelscited by 2,537
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  7. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  8. 2021ViViT: A Video Vision Transformercited by 1,610
  9. 2014DeepWalk: Online Learning of Social Representationscited by 1,543
  10. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  11. 2023PaLM 2 Technical Reportcited by 1,154
  12. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  13. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  14. 2020Benchmarking Graph Neural Networkscited by 672
  15. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
  16. 2023Symbolic Discovery of Optimization Algorithmscited by 423
  17. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 271
  18. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 249
  19. 2022DyLoRA: Parameter Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptationcited by 229
  20. 2023Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessmentcited by 227
  21. 2023Language is All a Graph Needscited by 219
  22. 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 209
  23. 2021Self-Supervised Learning of Graph Neural Networks: A Unified Reviewcited by 184
  24. 2022Sign and Basis Invariant Networks for Spectral Graph Representation Learningcited by 176
  25. 2023Talk like a Graph: Encoding Graphs for Large Language Modelscited by 175
  26. 2020Scaling Graph Neural Networks with Approximate PageRankcited by 174
  27. 2020Machine Learning on Graphs: A Model and Comprehensive Taxonomycited by 156
  28. 2021On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptationcited by 147
  29. 2022KronA: Parameter Efficient Tuning with Kronecker Adaptercited by 130
  30. 2021Breaking the Limits of Message Passing Graph Neural Networkscited by 101
  31. 2022Equivariant and Stable Positional Encoding for More Powerful Graph Neural Networkscited by 94
  32. 2020Masking as an Efficient Alternative to Finetuning for Pretrained Language Modelscited by 92
  33. 2023GPT-4 Doesn't Know It's Wrong: An Analysis of Iterative Prompting for Reasoning Problemscited by 88
  34. 2023Swallowing the Bitter Pill: Simplified Scalable Conformer Generationcited by 55
  35. 2022Standing on the Shoulders of Giant Frozen Language Modelscited by 33
  36. 2018SGR: Self-Supervised Spectral Graph Representation Learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 34; 2025: 45; 2026: 33.

112 papers.

Sort:

By area:

  1. 2024G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answeringcited by 217
  2. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  3. 2024Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoningcited by 83
  4. 2024Understanding Transformer Reasoning Capabilities via Graph Algorithmscited by 64
  5. 2024GraphArena: Evaluating and Exploring Large Language Models on Graph Computationcited by 48
  6. 2024Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insightscited by 44
  7. 2024Can Graph Learning Improve Planning in LLM-based Agents?cited by 42
  8. 2024Fully-inductive Node Classification on Arbitrary Graphscited by 36
  9. 2024Graph Machine Learning in the Era of Large Language Models (LLMs)cited by 33
  10. 2024GraphWiz: An Instruction-Following Language Model for Graph Problemscited by 32
  11. 2025Relational Transformer: Toward Zero-Shot Foundation Models for Relational Datacited by 32
  12. 2024How Do Large Language Models Understand Graph Patterns? A Benchmark for Graph Pattern Comprehensioncited by 30
38 more
  1. 2024Network Formation and Dynamics Among Multi-LLMscited by 28
  2. 2024Scalable and Accurate Graph Reasoning with LLM-based Multi-Agentscited by 21
  3. 2025When Do LLMs Help With Node Classification? A Comprehensive Analysiscited by 21
  4. 2024GraphInsight: Unlocking Insights in Large Language Models for Graph Structure Understandingcited by 18
  5. 2025GraphEval: A Lightweight Graph-Based LLM Framework for Idea Evaluationcited by 17
  6. 2025RAS: Retrieval-And-Structuring for Knowledge-Intensive LLM Generationcited by 16
  7. 2025RTL++: Graph-enhanced LLM for RTL Code Generationcited by 16
  8. 2024CATP-LLM: Empowering Large Language Models for Cost-Aware Tool Planningcited by 14
  9. 2025Multimodal Medical Code Tokenizercited by 14
  10. 2025G1: Teaching LLMs to Reason on Graphs with Reinforcement Learningcited by 14
  11. 2024StrucText-Eval: Evaluating Large Language Model's Reasoning Ability in Structure-Rich Textcited by 13
  12. 2025Graph2text or Graph2token: A Perspective of Large Language Models for Graph Learningcited by 12
  13. 2024Taxonomy Tree Generation from Citation Graphcited by 11
  14. 2024Node Identifiers: Compact, Discrete Representations for Efficient Graph Learningcited by 10
  15. 2024Revisiting the Graph Reasoning Ability of Large Language Models: Case Studies in Translation, Connectivity and Shortest Pathcited by 10
  16. 2024Graph Learning in the Era of LLMs: A Survey from the Perspective of Data, Models, and Taskscited by 10
  17. 2025Contextualizing biological perturbation experiments through languagecited by 10
  18. 2024A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspectivecited by 9
  19. 2025GraphThought: Graph Combinatorial Optimization with Thought Generationcited by 9
  20. 2025Multi-agent systems for chemical engineering: A review and perspectivecited by 9
  21. 2024Lost-in-Distance: Impact of Contextual Proximity on LLM Performance in Graph Taskscited by 8
  22. 2025Model Generalization on Text Attribute Graphs: Principles with Large Language Modelscited by 8
  23. 2025LightPROF: A Lightweight Reasoning Framework for Large Language Model on Knowledge Graphcited by 8
  24. 2024rLLM: Relational Table Learning with LLMscited by 7
  25. 2024CodeGraph: Enhancing Graph Reasoning of LLMs with Codecited by 6
  26. 2024Best of Both Worlds: Advantages of Hybrid Graph Sequence Modelscited by 6
  27. 2025K-Paths: Reasoning over Graph Paths for Drug Repurposing and Drug Interaction Predictioncited by 6
  28. 2025Graph-KV: Breaking Sequence via Injecting Structural Biases into Large Language Modelscited by 6
  29. 2025MIXRAG : Mixture-of-Experts Retrieval-Augmented Generation for Textual Graph Understanding and Question Answeringcited by 6
  30. 2025Rethinking and Benchmarking Large Language Models for Graph Reasoningcited by 6
  31. 2025Semantic Refinement with LLMs for Graph Representationscited by 6
  32. 2024Graph Reasoning with Large Language Models via Pseudo-code Promptingcited by 5
  33. 2024GUNDAM: Aligning Large Language Models with Graph Understandingcited by 5
  34. 2024Graph Linearization Methods for Reasoning on Graphs with Large Language Modelscited by 5
  35. 2025Injecting Knowledge Graphs into Large Language Modelscited by 5
  36. 2026Billion-Scale Graph Foundation Modelscited by 5
  37. 2024How to Make LMs Strong Node Classifiers?cited by 4
  38. 2024Towards Graph Foundation Models: A Study on the Generalization of Positional and Structural Encodingscited by 4

50 most cited of 112 All 112 (JSON)