mlsanityData to 7 Oct 2026

GraphWiz: An Instruction-Following Language Model for Graph Problems

arXiv 2402.16029 ↗First posted 25 Feb 2024cs.CLcited by 32

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

also in 1 more
Abstract (arXiv)

Large language models (LLMs) have achieved impressive success across several fields, but their proficiency in understanding and resolving complex graph problems is less explored. To bridge this gap, we introduce GraphInstruct, a novel and comprehensive instruction-tuning dataset designed to equip language models with the ability to tackle a broad spectrum of graph problems using explicit reasoning paths. Utilizing GraphInstruct, we build GraphWiz, an open-source language model capable of resolving various graph problem types while generating clear reasoning processes. To enhance the model's capability and reliability, we incorporate the Direct Preference Optimization (DPO) framework into the graph problem-solving context. The enhanced model, GraphWiz-DPO, achieves an average accuracy of 65% across nine tasks with different complexity levels, surpassing GPT-4 which has an average accuracy of 43.8%. Moreover, our research delves into the delicate balance between training data volume and model performance, highlighting the potential for overfitting with increased data. We also explore the transferability of the model's reasoning ability across different graph tasks, indicating the model's adaptability and practical application potential. Our investigation offers a new blueprint and valuable insights for developing LLMs specialized in graph reasoning and problem-solving.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 48 of its 62 references that are in the map. First posted on arXiv 25 Feb 2024.

Builds on

48 of 62 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignment13 Feb
  2. 2023GraphLLM: Boosting Graph Reasoning Ability of Large Language Model9 Oct
  3. 2023GraphText: Graph Reasoning in Text Space2 Oct
  4. 2023Can Language Models Solve Graph Problems in Natural Language?17 May

Other references · 44

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  7. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  8. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  9. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  10. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  11. 2017Graph Attention Networkscited by 8,854
  12. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
32 more
  1. 2023Segment Anythingcited by 8,158
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  5. 2018How Powerful are Graph Neural Networks?cited by 4,355
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  7. 2023Mistral 7Bcited by 3,065
  8. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  9. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  10. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  11. 2024Mixtral of Expertscited by 1,785
  12. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  13. 2023PaLM 2 Technical Reportcited by 1,154
  14. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  15. 2022PAL: Program-aided Language Modelscited by 635
  16. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
  17. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 271
  18. 2023Secrets of RLHF in Large Language Models Part I: PPOcited by 226
  19. 2021GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoningcited by 224
  20. 2023GraphGPT: Graph Instruction Tuning for Large Language Modelscited by 223
  21. 2023Language is All a Graph Needscited by 219
  22. 2023Talk like a Graph: Encoding Graphs for Large Language Modelscited by 175
  23. 2024The Impact of Reasoning Step Length on Large Language Modelscited by 152
  24. 2023Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learningcited by 132
  25. 2024Let Your Graph Do the Talking: Encoding Structured Data for LLMscited by 112
  26. 2023A Survey of Graph Meets Large Language Model: Progress and Future Directionscited by 84
  27. 2023Label-free Node Classification on Graphs with Large Language Models (LLMS)cited by 72
  28. 2023Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observationscited by 58
  29. 2021Self-supervised Contrastive Cross-Modality Representation Learning for Spoken Question Answeringcited by 39
  30. 2022End-to-end Spoken Conversational Question Answering: Task, Dataset and Modelcited by 37
  31. 2023A Fused Gromov-Wasserstein Framework for Unsupervised Knowledge Graph Entity Alignmentcited by 16
  32. 2023From Good to Great: Improving Math Reasoning with Tool-Augmented Interleaf Promptingcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 14; 2026: 3.

32 papers.

Sort:

By area:

  1. 2025A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Modelscited by 85
  2. 2024A Survey of Large Language Models for Graphscited by 71
  3. 2024Parameter-Efficient Fine-Tuning with Discrete Fourier Transformcited by 67
  4. 2024GraphArena: Evaluating and Exploring Large Language Models on Graph Computationcited by 48
  5. 2024Can LLM Graph Reasoning Generalize beyond Pattern Memorization?cited by 46
  6. 2024GLBench: A Comprehensive Benchmark for Graph with Large Language Modelscited by 36
  7. 2024Graph Machine Learning in the Era of Large Language Models (LLMs)cited by 33
  8. 2025Graph-based Approaches and Functionalities in Retrieval-Augmented Generation: A Comprehensive Surveycited by 18
  9. 2025GRAPHGPT-O: Synergistic Multimodal Comprehension and Generation on Graphscited by 17
  10. 2024LangGFM: A Large Language Model Alone Can be a Powerful Graph Foundation Modelcited by 13
  11. 2024GCoder: Improving Large Language Model for Generalized Graph Problem Solvingcited by 11
  12. 2024Revisiting the Graph Reasoning Ability of Large Language Models: Case Studies in Translation, Connectivity and Shortest Pathcited by 10
20 more
  1. 2024A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspectivecited by 9
  2. 2024PortLLM: Personalizing Evolving Large Language Models with Training-Free and Portable Model Patchescited by 7
  3. 2024CodeGraph: Enhancing Graph Reasoning of LLMs with Codecited by 6
  4. 2025Graph-KV: Breaking Sequence via Injecting Structural Biases into Large Language Modelscited by 6
  5. 2024GUNDAM: Aligning Large Language Models with Graph Understandingcited by 5
  6. 2025NPG-Muse: Scaling Long Chain-of-Thought Reasoning with NP-Hard Graph Problemscited by 5
  7. 2025Large Language Models Meet Text-Attributed Graphs: A Survey of Integration Frameworks and Applicationscited by 5
  8. 2025GraphChain: Large Language Models for Large-scale Graph Analysis via Tool Chainingcited by 5
  9. 2025Chain of Execution Supervision Promotes General Reasoning in Large Language Modelscited by 4
  10. 2026DynamicGTR: Leveraging Graph Topology Representation Preferences to Boost VLM Capabilities on Graph QAscited by 4
  11. 2024A Hierarchical Language Model For Interpretable Graph Reasoningcited by 3
  12. 2025Integrating Structural and Semantic Signals in Text-Attributed Graphs with BiGTexcited by 3
  13. 2024Joint Embeddings for Graph Instruction Tuningcited by 2
  14. 2025Improving LLMs' Generalized Reasoning Abilities by Graph Problemscited by 1
  15. 2025Harnessing Adaptive Topology Representations for Zero-Shot Graph Question Answeringcited by 1
  16. 2026GFMate: Empowering Graph Foundation Models with Test-time Prompt Tuningcited by 1
  17. 2025PrefixAgent: An LLM-Powered Design Framework for Efficient Prefix Adder Optimizationcited by 0
  18. 2025GraphShaper: Geometry-aware Alignment for Improving Transfer Learning in Text-Attributed Graphscited by 0
  19. 2025Efficiently Learning Branching Networks for Multitask Algorithmic Reasoningcited by 0
  20. 2026AdaSTORM: Scaling LLM Reasoning on Dynamic Graphs via Adaptive Spatio-Temporal Multi-Agent Collaborationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024GraphInstruct: Empowering Large Language Models with Graph Understanding and Reasoning Capability
  2. 2025Rethinking and Benchmarking Large Language Models for Graph Reasoning
  3. 2026Are Large Language Models Suitable for Graph Computation? Progress and Prospects
  4. 2025G1: Teaching LLMs to Reason on Graphs with Reinforcement Learning
  5. 2024GraphTool-Instruction: Revolutionizing Graph Reasoning in LLMs through Decomposed Subtask Instruction
  6. 2026GraphDancer: Training LLMs to Explore and Reason over Graphs via Two-Stage Curriculum Post-Training
  7. 2026Unified Multi-Dimensional Benchmark for Complex Graph Reasoning in Large Language Models
  8. 2025Exploring Graph Learning Tasks with Pure LLMs: A Comprehensive Benchmark and Investigation