mlsanityData to 7 Oct 2026

Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPT

arXiv 2304.11116 ↗First posted 10 Apr 2023cs.AI cs.LGcited by 78

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

In this paper, we aim to develop a large language model (LLM) with the reasoning ability on complex graph data. Currently, LLMs have achieved very impressive performance on various natural language learning tasks, extensions of which have also been applied to study the vision tasks with multi-modal data. However, when it comes to the graph learning tasks, existing LLMs present very serious flaws due to their several inherited weaknesses in performing {multi-step logic reasoning}, {precise mathematical calculation} and {perception about the spatial and temporal factors}. To address such challenges, in this paper, we will investigate the principles, methodologies and algorithms to empower existing LLMs with graph reasoning ability, which will have tremendous impacts on the current research of both LLMs and graph learning. Inspired by the latest ChatGPT and Toolformer models, we propose the Graph-ToolFormer (Graph Reasoning oriented Toolformer) framework to teach LLMs themselves with prompts augmented by ChatGPT to use external graph reasoning API tools. Specifically, we will investigate to teach Graph-ToolFormer to handle various graph data reasoning tasks in this paper, including both (1) very basic graph data loading and graph property reasoning tasks, ranging from simple graph order and size to the graph diameter and periphery, and (2) more advanced reasoning tasks on real-world graph data, such as bibliographic networks, protein molecules, sequential recommender systems, social networks and knowledge graphs.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 38 of its 71 references that are in the map. First posted on arXiv 10 Apr 2023.

Builds on

38 of 71 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  9. 2017Graph Attention Networkscited by 8,854
  10. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  11. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  12. 2021Zero-Shot Text-to-Image Generationcited by 4,276
26 more
  1. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  2. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  3. 2018Deep contextualized word representationscited by 2,285
  4. 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,099
  5. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  6. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  7. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  8. 2015Image-based Recommendations on Styles and Substitutescited by 702
  9. 2020A Survey on Knowledge Graphs: Representation, Acquisition and Applicationscited by 643
  10. 2019Graph Transformer Networkscited by 520
  11. 2023Augmented Language Models: a Surveycited by 398
  12. 20218-bit Optimizers via Block-wise Quantizationcited by 395
  13. 2019Multi-Task Deep Neural Networks for Natural Language Understandingcited by 349
  14. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
  15. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 324
  16. 2020GPT-GNN: Generative Pre-Training of Graph Neural Networkscited by 302
  17. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 290
  18. 2020Graph-Bert: Only Attention is Needed for Learning Graph Representationscited by 176
  19. 2015A Survey of Heterogeneous Information Network Analysiscited by 109
  20. 2019AdaGCN: Adaboosting Graph Convolutional Networks into Deep Modelscited by 37
  21. 2018Spectral Collaborative Filteringcited by 34
  22. 2019GResNet: Graph Residual Network for Reviving Deep GNNs from Suspended Animationcited by 21
  23. 2019Graph Neural Networks for Small Graph and Giant Network Representation Learning: An Overviewcited by 3
  24. 2020Segmented Graph-Bert for Graph Instance Modelingcited by 3
  25. 2019Residual or Gate? Towards Deeper Graph Neural Networks for Inductive Graph Representation Learningcited by 2
  26. 2019DeGNN: Characterizing and Improving Graph Neural Networks with Graph Decompositioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 23; 2024: 30; 2025: 20; 2026: 5.

78 papers.

Sort:

By area:

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  2. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
  3. 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 289
  4. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  5. 2023ToolQA: A Dataset for LLM Question Answering with External Toolscited by 273
  6. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 271
  7. 2024Tool Learning with Large Language Models: A Surveycited by 221
  8. 2023Language is All a Graph Needscited by 219
  9. 2024G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answeringcited by 217
  10. 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 209
  11. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 190
  12. 2023Recommender Systems in the Era of Large Language Models (LLMs)cited by 188
38 more
  1. 2023Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learningcited by 132
  2. 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 109
  3. 2023GraphLLM: Boosting Graph Reasoning Ability of Large Language Modelcited by 105
  4. 2025Agentic Large Language Models, a surveycited by 105
  5. 2023ToolChain*: Efficient Action Space Navigation in Large Language Models with A* Searchcited by 103
  6. 2024Demystifying Chains, Trees, and Graphs of Thoughtscited by 63
  7. 2024InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignmentcited by 56
  8. 2024A Comprehensive Survey of Foundation Models in Medicinecited by 56
  9. 2023KnowledGPT: Enhancing Large Language Models with Retrieval and Storage Access on Knowledge Basescited by 52
  10. 2023Amazon-M2: A Multilingual Multi-locale Shopping Session Dataset for Recommendation and Text Generationcited by 42
  11. 2023Evaluating Large Language Models on Graphs: Performance Insights and Comparative Analysiscited by 35
  12. 2024LLMs Meet Multimodal Generation and Editing: A Surveycited by 35
  13. 2024Graph Machine Learning in the Era of Large Language Models (LLMs)cited by 33
  14. 2023Large Language Models as Topological Structure Enhancers for Text-Attributed Graphscited by 27
  15. 2023Graph Meets LLMs: Towards Large Graph Modelscited by 19
  16. 2023Holistic Evaluation of GPT-4V for Biomedical Imagingcited by 19
  17. 2023User Modeling in the Era of Large Language Models: Current Research and Future Directionscited by 18
  18. 2024A Survey of Large Language Models on Generative Graph Analytics: Query, Learning, and Applicationscited by 17
  19. 2025GRAPHGPT-O: Synergistic Multimodal Comprehension and Generation on Graphscited by 17
  20. 2024WildfireGPT: Tailored Large Language Model for Wildfire Analysiscited by 15
  21. 2024Exploring the Potential of Large Language Models in Graph Generationcited by 15
  22. 2025Graph RAG-Tool Fusioncited by 15
  23. 2025Graph2text or Graph2token: A Perspective of Large Language Models for Graph Learningcited by 12
  24. 2024Can LLMs Understand Computer Networks? Towards a Virtual System Administratorcited by 11
  25. 2025Exploring Graph Learning Tasks with Pure LLMs: A Comprehensive Benchmark and Investigationcited by 11
  26. 2024Similarity-based Neighbor Selection for Graph LLMscited by 10
  27. 2023LLM4Causal: Democratized Causal Tools for Everyone via Large Language Modelcited by 9
  28. 2024"Don't forget to put the milk back!" Dataset for Enabling Embodied Agents to Detect Anomalous Situationscited by 9
  29. 2024MaFeRw: Query Rewriting with Multi-Aspect Feedbacks for Retrieval-Augmented Large Language Modelscited by 9
  30. 2023Testing the Depth of ChatGPT's Comprehension via Cross-Modal Tasks Based on ASCII-Art: GPT3.5's Abilities in Regard to Recognizing and Generating ASCII-Art Are Not Totally Lackingcited by 7
  31. 2024A Unified Taxonomy-Guided Instruction Tuning Framework for Entity Set Expansion and Taxonomy Expansioncited by 7
  32. 2024SoAy: A Solution-based LLM API-using Methodology for Academic Information Seekingcited by 7
  33. 2025GraphChain: Large Language Models for Large-scale Graph Analysis via Tool Chainingcited by 5
  34. 2024Knowledge Graph Modeling-Driven Large Language Model Operating System (LLM OS) for Task Automation in Process Engineering Problem-Solvingcited by 4
  35. 2024How to Make LMs Strong Node Classifiers?cited by 4
  36. 2025GraphRunner: A Multi-Stage Framework for Efficient and Accurate Graph-Based Retrievalcited by 4
  37. 2025Agentic AI for autonomous anomaly management in complex systemscited by 4
  38. 2025Actions Speak Louder than Prompts: A Large-Scale Study of LLMs for Graph Inferencecited by 4

50 most cited of 78 All 78 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Can LLMs perform structured graph reasoning?
  2. 2024Graph Reasoning with Large Language Models via Pseudo-code Prompting
  3. 2025Rethinking and Benchmarking Large Language Models for Graph Reasoning
  4. 2025Less is More: Learning Graph Tasks with Just LLMs
  5. 2023GraphReason: Enhancing Reasoning Capabilities of Large Language Models through A Graph-Based Verification Approach
  6. 2024CodeGraph: Enhancing Graph Reasoning of LLMs with Code
  7. 2025Learn to Think: Bootstrapping LLM Reasoning Capability Through Graph Representation Learning
  8. 2024GraphTool-Instruction: Revolutionizing Graph Reasoning in LLMs through Decomposed Subtask Instruction