# ChatRule: Mining Logical Rules with Large Language Models for Knowledge Graph Reasoning

arXiv: https://arxiv.org/abs/2309.01538 · HTML: https://mlsanity.com/paper/2309.01538/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 22 of its 39 references that are in the map.
- First posted on arXiv 4 Sep 2023.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)

## Builds on (most cited first, 22 of 22)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023)
- [DRUM: End-To-End Differentiable Rule Mining On Knowledge Graphs](https://mlsanity.com/paper/1911.00055/) (2019)
- [RNNLogic: Learning Logic Rules for Reasoning on Knowledge Graphs](https://mlsanity.com/paper/2010.04029/) (2020)
- [Evaluating Commonsense in Pre-trained Language Models](https://mlsanity.com/paper/1911.11931/) (2019)
- [Probabilistic Logic Neural Networks for Reasoning](https://mlsanity.com/paper/1906.08495/) (2019)
- [Improving Question Answering by Commonsense-Based Pre-Training](https://mlsanity.com/paper/1809.03568/) (2018)
- [TLogic: Temporal Logical Rules for Explainable Link Forecasting on Temporal Knowledge Graphs](https://mlsanity.com/paper/2112.08025/) (2021)
- [Learn to Explain Efficiently via Neural Logic Inductive Learning](https://mlsanity.com/paper/1910.02481/) (2019)
- [Neural Multi-Hop Reasoning With Logical Rules on Biomedical Knowledge Graphs](https://mlsanity.com/paper/2103.10367/) (2021)
- [Neural Compositional Rule Learning for Knowledge Graph Reasoning](https://mlsanity.com/paper/2303.03581/) (2023)
- [A Survey on Temporal Knowledge Graph Completion: Taxonomy, Progress, and Prospects](https://mlsanity.com/paper/2308.02457/) (2023)
- [Ruleformer: Context-aware Differentiable Rule Mining over Knowledge Graph](https://mlsanity.com/paper/2209.05815/) (2022)
- [R5: Rule Discovery with Reinforced and Recurrent Relational Reasoning](https://mlsanity.com/paper/2205.06454/) (2022)

## Built on by (most cited first, 26 of 26)

- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning](https://mlsanity.com/paper/2310.01061/) (2023)
- [Towards Self-Interpretable Graph-Level Anomaly Detection](https://mlsanity.com/paper/2310.16520/) (2023)
- [Large Language Models-guided Dynamic Adaptation for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2405.14170/) (2024)
- [Graph Machine Learning in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2404.14928/) (2024)
- [WALL-E: World Alignment by Rule Learning Improves World Model-based LLM Agents](https://mlsanity.com/paper/2410.07484/) (2024)
- [Graph Learning under Distribution Shifts: A Comprehensive Survey on Domain Adaptation, Out-of-distribution, and Continual Learning](https://mlsanity.com/paper/2402.16374/) (2024)
- [Research Trends for the Interplay between Large Language Models and Knowledge Graphs](https://mlsanity.com/paper/2406.08223/) (2024)
- [Evaluating Deep Unlearning in Large Language Models](https://mlsanity.com/paper/2410.15153/) (2024)
- [Online GNN Evaluation Under Test-time Graph Distribution Shifts](https://mlsanity.com/paper/2403.09953/) (2024)
- [HiBench: Benchmarking LLMs Capability on Hierarchical Structure Reasoning](https://mlsanity.com/paper/2503.00912/) (2025)
- [Improving Complex Reasoning over Knowledge Graph with Logic-Aware Curriculum Tuning](https://mlsanity.com/paper/2405.01649/) (2024)
- [Rethinking Regularization Methods for Knowledge Graph Completion](https://mlsanity.com/paper/2505.23442/) (2025)
- [RuleReasoner: Reinforced Rule-based Reasoning via Domain-aware Dynamic Sampling](https://mlsanity.com/paper/2506.08672/) (2025)
- [RuAG: Learned-rule-augmented Generation for Large Language Models](https://mlsanity.com/paper/2411.03349/) (2024)
- [Large Language Models Meet Graph Neural Networks: A Perspective of Graph Mining](https://mlsanity.com/paper/2412.19211/) (2024)
- [Validating ChatGPT Facts through RDF Knowledge Graphs and Sentence Similarity](https://mlsanity.com/paper/2311.04524/) (2023)
- [Neurosymbolic Methods for Rule Mining](https://mlsanity.com/paper/2408.05773/) (2024)
- [Is Large Language Model Good at Triple Set Prediction? An Empirical Study](https://mlsanity.com/paper/2412.18443/) (2024)
- [SHERLOCK: Towards Dynamic Knowledge Adaptation in LLM-enhanced E-commerce Risk Management](https://mlsanity.com/paper/2510.08948/) (2025)
- [Efficient Relational Context Perception for Knowledge Graph Completion](https://mlsanity.com/paper/2501.00397/) (2024)
- [A Survey of Task-Oriented Knowledge Graph Reasoning: Status, Applications, and Prospects](https://mlsanity.com/paper/2506.11012/) (2025)
- [Driver Assistant: Persuading Drivers to Adjust Secondary Tasks Using Large Language Models](https://mlsanity.com/paper/2508.05238/) (2025)
- [Bridging Graph Structure and Knowledge-Guided Editing for Interpretable Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2601.21978/) (2026)
- [Knowledge Graphs Meet Graph Neural Networks: A Comprehensive Survey](https://mlsanity.com/paper/2607.09666/) (2026)
- [Structure-Internalized Rule Language Model for Faithful Knowledge Graph Reasoning](https://mlsanity.com/paper/2608.17443/) (2026)

## Abstract (arXiv)

Logical rules are essential for uncovering the logical connections between relations, which could improve reasoning performance and provide interpretable results on knowledge graphs (KGs). Although there have been many efforts to mine meaningful logical rules over KGs, existing methods suffer from computationally intensive searches over the rule space and a lack of scalability for large-scale KGs. Besides, they often ignore the semantics of relations which is crucial for uncovering logical connections. Recently, large language models (LLMs) have shown impressive performance in the field of natural language processing and various applications, owing to their emergent ability and generalizability. In this paper, we propose a novel framework, ChatRule, unleashing the power of large language models for mining logical rules over knowledge graphs. Specifically, the framework is initiated with an LLM-based rule generator, leveraging both the semantic and structural information of KGs to prompt LLMs to generate logical rules. To refine the generated rules, a rule ranking module estimates the rule quality by incorporating facts from existing KGs. Last, the ranked rules can be used to conduct reasoning over KGs. ChatRule is evaluated on four large-scale KGs, w.r.t. different rule quality metrics and downstream tasks, showing the effectiveness and scalability of our method.