mlsanityData to 7 Oct 2026

InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignment

arXiv 2402.08785 ↗First posted 13 Feb 2024cs.CLcited by 56

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

also in 1 more
Abstract (arXiv)

Do current large language models (LLMs) better solve graph reasoning and generation tasks with parameter updates? In this paper, we propose InstructGraph, a framework that empowers LLMs with the abilities of graph reasoning and generation by instruction tuning and preference alignment. Specifically, we first propose a structured format verbalizer to unify all graph data into a universal code-like format, which can simply represent the graph without any external graph-specific encoders. Furthermore, a graph instruction tuning stage is introduced to guide LLMs in solving graph reasoning and generation tasks. Finally, we identify potential hallucination problems in graph tasks and sample negative instances for preference alignment, the target of which is to enhance the output's reliability of the model. Extensive experiments across multiple graph-centric tasks exhibit that InstructGraph can achieve the best performance and outperform GPT-4 and LLaMA2 by more than 13% and 38%, respectively.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 53 of its 87 references that are in the map. First posted on arXiv 13 Feb 2024.

Builds on

53 of 87 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023KG-GPT: A General Framework for Reasoning on Knowledge Graphs Using Large Language Models17 Oct
  2. 2023GraphText: Graph Reasoning in Text Space2 Oct
  3. 2023Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning2 Oct
  4. 2023Can Language Models Solve Graph Problems in Natural Language?17 May
  5. 2023Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPT10 Apr

Other references · 48

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  9. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  10. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  11. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  12. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
36 more
  1. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  2. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  3. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  4. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  5. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  6. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 641
  7. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  8. 2016Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filteringcited by 517
  9. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 509
  10. 2019KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representationcited by 356
  11. 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 339
  12. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 319
  13. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimizationcited by 223
  14. 2023Language is All a Graph Needscited by 219
  15. 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 209
  16. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 200
  17. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 190
  18. 2020Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Basescited by 169
  19. 2023InstructUIE: Multi-task Instruction Tuning for Unified Information Extractioncited by 167
  20. 2017Interactive Learning from Policy-Dependent Human Feedbackcited by 155
  21. 2022On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?cited by 137
  22. 2023Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigationcited by 123
  23. 2022Evaluating the Factual Consistency of Large Language Models Through News Summarizationcited by 90
  24. 2023At Which Training Stage Does Code Data Help LLMs Reasoning?cited by 82
  25. 2023Boosting Language Models Reasoning with Chain-of-Knowledge Promptingcited by 77
  26. 2024If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agentscited by 77
  27. 2018An Interpretable Reasoning Network for Multi-Relation Question Answeringcited by 47
  28. 2023What Makes Good In-context Demonstrations for Code Intelligence Tasks with LLMs?cited by 47
  29. 2023PiVe: Prompting with Iterative Verification Improving Graph-based Generative Capability of LLMscited by 44
  30. 2022A Decade of Knowledge Graphs in Natural Language Processing: A Surveycited by 40
  31. 2023LLM4DyG: Can Large Language Models Solve Spatial-Temporal Problems on Dynamic Graphs?cited by 40
  32. 2023Symbol-LLM: Towards Foundational Symbol-centric Interface For Large Language Modelscited by 38
  33. 2023Disentangled Representation Learning with Large Language Models for Text-Attributed Graphscited by 36
  34. 2023Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Reviewcited by 32
  35. 2021EventNarrative: A large-scale Event-centric Dataset for Knowledge Graph-to-Text Generationcited by 14
  36. 2023InstructIE: A Bilingual Instruction-based Information Extraction Datasetcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 20; 2025: 16; 2026: 20.

56 papers.

Sort:

By area:

  1. 2024A Survey of Large Language Models for Graphscited by 71
  2. 2025CAD-Llama: Leveraging Large Language Models for Computer-Aided Design Parametric 3D Model Generationcited by 65
  3. 2024GraphArena: Evaluating and Exploring Large Language Models on Graph Computationcited by 48
  4. 2024Can LLM Graph Reasoning Generalize beyond Pattern Memorization?cited by 46
  5. 2024CoRAL: Collaborative Retrieval-Augmented Large Language Models Improve Long-tail Recommendationcited by 44
  6. 2024Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insightscited by 44
  7. 2024Knowledge Graph Large Language Model (KG-LLM) for Link Predictioncited by 36
  8. 2024GraphWiz: An Instruction-Following Language Model for Graph Problemscited by 32
  9. 2024Let's Ask GNN: Empowering Large Language Model for Graph In-Context Learningcited by 21
  10. 2025LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Modelscited by 21
  11. 2024Personalized Multimodal Large Language Models: A Surveycited by 19
  12. 2025GRAPHGPT-O: Synergistic Multimodal Comprehension and Generation on Graphscited by 17
38 more
  1. 2026MemRec: Collaborative Memory-Augmented Agentic Recommender Systemcited by 15
  2. 2024LangGFM: A Large Language Model Alone Can be a Powerful Graph Foundation Modelcited by 13
  3. 2024UniGAD: Unifying Multi-level Graph Anomaly Detectioncited by 13
  4. 2026Graph is a Substrate Across Data Modalitiescited by 13
  5. 2024OCEAN: Offline Chain-of-thought Evaluation and Alignment in Large Language Modelscited by 12
  6. 2026Skill-R1: Agent Skill Evolution via Reinforcement Learningcited by 12
  7. 2024GCoder: Improving Large Language Model for Generalized Graph Problem Solvingcited by 11
  8. 2025Exploring Graph Learning Tasks with Pure LLMs: A Comprehensive Benchmark and Investigationcited by 11
  9. 2024A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspectivecited by 9
  10. 2025GraphCogent: Mitigating LLMs' Working Memory Constraints via Multi-Agent Collaboration in Complex Graph Understandingcited by 9
  11. 2024CoMMIT: Coordinated Multimodal Instruction Tuningcited by 8
  12. 2024Effective Instruction Parsing Plugin for Complex Logical Query Answering on Knowledge Graphscited by 8
  13. 2026Toward Graph-Tokenizing Large Language Models with Reconstructive Graph Instruction Tuningcited by 7
  14. 2024Thinking with Knowledge Graphs: Enhancing LLM Reasoning Through Structured Datacited by 6
  15. 2024Investigating Instruction Tuning Large Language Models on Graphscited by 5
  16. 2025Towards Graph Foundation Models: A Transferability Perspectivecited by 5
  17. 2025GraphChain: Large Language Models for Large-scale Graph Analysis via Tool Chainingcited by 5
  18. 2026GNN-as-Judge: Unleashing the Power of LLMs for Graph Learning with GNN Feedbackcited by 4
  19. 2024A Hierarchical Language Model For Interpretable Graph Reasoningcited by 3
  20. 2025Language Models, Graph Searching, and Supervision Adulteration: When More Supervision is Less and How to Make More Morecited by 3
  21. 2025Integrating Structural and Semantic Signals in Text-Attributed Graphs with BiGTexcited by 3
  22. 2025Generalizable LLM Learning of Graph Synthetic Data with Post-training Alignmentcited by 3
  23. 2025From Sequence to Structure: Uncovering Substructure Reasoning in Transformerscited by 3
  24. 2025A Survey of Cross-domain Graph Learning: Progress and Future Directionscited by 2
  25. 2025GILT: An LLM-Free, Tuning-Free Graph Foundational Model for In-Context Learningcited by 2
  26. 2026<SOG_k>: One LLM Token for Explicit Graph Structural Understandingcited by 2
  27. 2026Revisiting Graph-Tokenizing Large Language Models: A Systematic Evaluation of Graph Token Understandingcited by 2
  28. 2024Can Large Language Models Act as Ensembler for Multi-GNNs?cited by 1
  29. 2025Improving LLMs' Generalized Reasoning Abilities by Graph Problemscited by 1
  30. 2025GLOW: Graph-Language Co-Encoding for Agentic Workflow Performance Predictioncited by 1
  31. 2026FARE: Fast-Slow Agentic Robotic Explorationcited by 1
  32. 2026Beyond One-Size-Fits-All: Adaptive Subgraph Denoising for Zero-Shot Graph Learning with Large Language Modelscited by 1
  33. 2026Integrating Graphs, Large Language Models, and Agents: Reasoning and Retrievalcited by 1
  34. 2026Surprisingly Simple and Effective Multi-Domain Graph Foundation Model through Graph-to-Table Alignmentcited by 1
  35. 2025Adaptive Tokenization: On the Hop-Overpriority Problem in Tokenized Graph Learning Modelscited by 0
  36. 2026Semi-supervised Instruction Tuning for Large Language Models on Text-Attributed Graphscited by 0
  37. 2026HopRank: Self-Supervised LLM Preference-Tuning on Graphs for Few-Shot Node Classificationcited by 0
  38. 2026SLASH the Sink: Sharpening Structural Attention Inside LLMscited by 0

50 most cited of 56 All 56 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024GraphInstruct: Empowering Large Language Models with Graph Understanding and Reasoning Capability
  2. 2024GraphTool-Instruction: Revolutionizing Graph Reasoning in LLMs through Decomposed Subtask Instruction
  3. 2025Scalability Matters: Overcoming Challenges in InstructGLM with Similarity-Degree-Based Sampling
  4. 2025Rethinking and Benchmarking Large Language Models for Graph Reasoning
  5. 2024Graph Reasoning with Large Language Models via Pseudo-code Prompting
  6. 2024Can LLMs perform structured graph reasoning?
  7. 2024Graph-oriented Instruction Tuning of Large Language Models for Generic Graph Mining
  8. 2025GraphOmni: A Comprehensive and Extensible Benchmark Framework for Large Language Models on Graph-theoretic Tasks