mlsanityData to 7 Oct 2026

A Hierarchical Language Model For Interpretable Graph Reasoning

arXiv 2410.22372 ↗First posted 29 Oct 2024cs.LG cs.AIcited by 3

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

Large language models (LLMs) are being increasingly explored for graph tasks. Despite their remarkable success in text-based tasks, LLMs' capabilities in understanding explicit graph structures remain limited, particularly with large graphs. In this work, we introduce Hierarchical Language Model for Graph (HLM-G), which employs a two-block architecture to capture node-centric local information and interaction-centric global structure, effectively enhancing graph structure understanding abilities. The proposed scheme allows LLMs to address various graph queries with high efficacy, efficiency, and robustness, while reducing computational costs on large-scale graph tasks. Furthermore, we demonstrate the interpretability of our model using intrinsic attention weights and established explainers. Comprehensive evaluations across diverse graph reasoning and real-world tasks of node, link, and graph-levels highlight the superiority of our method, marking a significant advancement in the application of LLMs to graph understanding.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 44 of its 93 references that are in the map. First posted on arXiv 29 Oct 2024.

Builds on

44 of 93 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024GraphInstruct: Empowering Large Language Models with Graph Understanding and Reasoning Capability7 Mar
  2. 2024GraphWiz: An Instruction-Following Language Model for Graph Problems25 Feb
  3. 2024InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignment13 Feb
  4. 2023GraphLLM: Boosting Graph Reasoning Ability of Large Language Model9 Oct
  5. 2023GraphText: Graph Reasoning in Text Space2 Oct

Other references · 39

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2024The Llama 3 Herd of Modelscited by 14,192
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  8. 2017Graph Attention Networkscited by 8,854
  9. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  10. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  11. 2018How Powerful are Graph Neural Networks?cited by 4,355
  12. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
27 more
  1. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  2. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  3. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
  4. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 428
  5. 2016Not Just a Black Box: Learning Important Features Through Propagating Activation Differencescited by 295
  6. 2023A Survey on Oversmoothing in Graph Neural Networkscited by 246
  7. 2023A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Modelscited by 244
  8. 2023One for All: Towards Training One Graph Model for All Classification Taskscited by 239
  9. 2023GraphGPT: Graph Instruction Tuning for Large Language Modelscited by 223
  10. 2023Language is All a Graph Needscited by 219
  11. 2024G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answeringcited by 217
  12. 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 209
  13. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 190
  14. 2024LLaGA: Large Language and Graph Assistantcited by 177
  15. 2023Talk like a Graph: Encoding Graphs for Large Language Modelscited by 175
  16. 2021Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Predictioncited by 137
  17. 2024AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformerscited by 116
  18. 2024Let Your Graph Do the Talking: Encoding Structured Data for LLMscited by 112
  19. 2024Evaluating the World Model Implicit in a Generative Modelcited by 107
  20. 2024Chain of Thoughtlessness? An Analysis of CoT in Planningcited by 100
  21. 2024A Survey of Large Language Models for Graphscited by 71
  22. 2024Understanding Transformer Reasoning Capabilities via Graph Algorithmscited by 64
  23. 2023Disentangled Representation Learning with Large Language Models for Text-Attributed Graphscited by 36
  24. 2023Can LLMs Effectively Leverage Graph Structural Information through Prompts, and Why?cited by 31
  25. 2023Beyond Text: A Deep Dive into Large Language Models' Ability on Understanding Graph Datacited by 25
  26. 2024Comparing Graph Transformers via Positional Encodingscited by 20
  27. 2022Efficient and effective training of language and graph neural network modelscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  2. 2026Testing LLM Arithmetic Reasoning Generalization with Automatic Numeric-Remapping Attackscited by 1
  3. 2026Are Large Language Models Suitable for Graph Computation? Progress and Prospectscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Revisiting the Graph Reasoning Ability of Large Language Models: Case Studies in Translation, Connectivity and Shortest Path
  2. 2024GUNDAM: Aligning Large Language Models with Graph Understanding
  3. 2025Rethinking and Benchmarking Large Language Models for Graph Reasoning
  4. 2024GraphInsight: Unlocking Insights in Large Language Models for Graph Structure Understanding
  5. 2026Detecting Differences Is Not Understanding Structure: Large Language Models Fail at Graph Isomorphism
  6. 2023Evaluating Large Language Models on Graphs: Performance Insights and Comparative Analysis
  7. 2026Teaching LLMs to See Graphs: Unifying Text and Structural Reasoning
  8. 2024Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs