# Collaborative Policy Learning for Open Knowledge Graph Reasoning

arXiv: https://arxiv.org/abs/1909.00230 · HTML: https://mlsanity.com/paper/1909.00230/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Aug 2019.

## Built on by (most cited first, 18 of 18)

- [Large Language Models for Generative Information Extraction: A Survey](https://mlsanity.com/paper/2312.17617/) (2023)
- [A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multimodal](https://mlsanity.com/paper/2212.05767/) (2022)
- [A Survey on Neural Open Information Extraction: Current Status and Future Directions](https://mlsanity.com/paper/2205.11725/) (2022)
- [Is Multi-Hop Reasoning Really Explainable? Towards Benchmarking Reasoning Interpretability](https://mlsanity.com/paper/2104.06751/) (2021)
- [Text-Augmented Open Knowledge Graph Completion via Pre-Trained Language Models](https://mlsanity.com/paper/2305.15597/) (2023)
- [Learning to Navigate Wikipedia by Taking Random Walks](https://mlsanity.com/paper/2211.00177/) (2022)
- [A Condensed Transition Graph Framework for Zero-shot Link Prediction with Large Language Models](https://mlsanity.com/paper/2402.10779/) (2024)
- [Knowledge Graph Reasoning with Self-supervised Reinforcement Learning](https://mlsanity.com/paper/2405.13640/) (2024)
- [Explainable Sparse Knowledge Graph Completion via High-order Graph Reasoning Network](https://mlsanity.com/paper/2207.07503/) (2022)
- [IntentDial: An Intent Graph based Multi-Turn Dialogue System with Reasoning Path Visualization](https://mlsanity.com/paper/2310.11818/) (2023)
- [Look Globally and Reason: Two-stage Path Reasoning over Sparse Knowledge Graphs](https://mlsanity.com/paper/2407.18556/) (2024)
- [EL4NER: Ensemble Learning for Named Entity Recognition via Multiple Small-Parameter Large Language Models](https://mlsanity.com/paper/2505.23038/) (2025)
- [VEM2L: A Plug-and-play Framework for Fusing Text and Structure Knowledge on Sparse Knowledge Graph Completion](https://mlsanity.com/paper/2207.01528/) (2022)
- [AgREE: Agentic Reasoning for Knowledge Graph Completion on Emerging Entities](https://mlsanity.com/paper/2508.04118/) (2025)
- [Knowledge Graph Completion with Text-aided Regularization](https://mlsanity.com/paper/2101.08962/) (2021)
- [APPTeK: Agent-Based Predicate Prediction in Temporal Knowledge Graphs](https://mlsanity.com/paper/2110.14284/) (2021)
- [Discover Important Paths in the Knowledge Graph Based on Dynamic Relation Confidence](https://mlsanity.com/paper/2211.00914/) (2022)
- [Emulating the Human Mind: A Neural-symbolic Link Prediction Model with Fast and Slow Reasoning and Filtered Rules](https://mlsanity.com/paper/2310.13996/) (2023)

## Abstract (arXiv)

In recent years, there has been a surge of interests in interpretable graph reasoning methods. However, these models often suffer from limited performance when working on sparse and incomplete graphs, due to the lack of evidential paths that can reach target entities. Here we study open knowledge graph reasoning---a task that aims to reason for missing facts over a graph augmented by a background text corpus. A key challenge of the task is to filter out "irrelevant" facts extracted from corpus, in order to maintain an effective search space during path inference. We propose a novel reinforcement learning framework to train two collaborative agents jointly, i.e., a multi-hop graph reasoner and a fact extractor. The fact extraction agent generates fact triples from corpora to enrich the graph on the fly; while the reasoning agent provides feedback to the fact extractor and guides it towards promoting facts that are helpful for the interpretable reasoning. Experiments on two public datasets demonstrate the effectiveness of the proposed approach. Source code and datasets used in this paper can be downloaded at https://github.com/shanzhenren/CPL