mlsanityData to 7 Oct 2026

Multi-Hop Knowledge Graph Reasoning with Reward Shaping

arXiv 1808.10568 ↗First posted 31 Aug 2018cs.AI cs.CL cs.LGcited by 102

Not on the map. Why?

Abstract (arXiv)

Multi-hop reasoning is an effective approach for query answering (QA) over incomplete knowledge graphs (KGs). The problem can be formulated in a reinforcement learning (RL) setup, where a policy-based agent sequentially extends its inference path until it reaches a target. However, in an incomplete KG environment, the agent receives low-quality rewards corrupted by false negatives in the training data, which harms generalization at test time. Furthermore, since no golden action sequence is used for training, the agent can be misled by spurious search trajectories that incidentally lead to the correct answer. We propose two modeling advances to address both issues: (1) we reduce the impact of false negative supervision by adopting a pretrained one-hop embedding model to estimate the reward of unobserved facts; (2) we counter the sensitivity to spurious paths of on-policy RL by forcing the agent to explore a diverse set of paths using randomly generated edge masks. Our approach significantly improves over existing path-based KGQA models on several benchmark datasets and is comparable or better than embedding-based models.

Key facts. Cited by 102 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 Aug 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 26; 2022: 24; 2023: 16; 2024: 17; 2025: 8; 2026: 11.

102 papers.

Sort:

By area:

  1. 2021Do Transformers Really Perform Bad for Graph Representation?cited by 318
  2. 2021Neural Bellman-Ford Networks: A General Graph Neural Network Framework for Link Predictioncited by 273
  3. 2022A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multimodalcited by 106
  4. 2021Improving Multi-hop Knowledge Base Question Answering by Learning Intermediate Supervision Signalscited by 100
  5. 2021ConE: Cone Embeddings for Multi-Hop Reasoning over Knowledge Graphscited by 78
  6. 2021TimeTraveler: Reinforcement Learning for Temporal Knowledge Graph Forecastingcited by 67
  7. 2022A*Net: A Scalable Path-based Reasoning Approach for Knowledge Graphscited by 49
  8. 2021Search from History and Reason for Future: Two-stage Reasoning on Temporal Knowledge Graphscited by 39
  9. 2023Neural Graph Reasoning: Complex Logical Query Answering Meets Graph Databasescited by 35
  10. 2022Answering Complex Logical Queries on Knowledge Graphs via Query Computation Tree Optimizationcited by 28
  11. 2021Neural Multi-Hop Reasoning With Logical Rules on Biomedical Knowledge Graphscited by 25
  12. 2022Query2Particles: Knowledge Graph Reasoning with Particle Embeddingscited by 24
38 more
  1. 2024Advancing Agentic Systems: Dynamic Task Decomposition, Tool Integration and Evaluation using Novel Metrics and Datasetcited by 24
  2. 2022GammaE: Gamma Embeddings for Logical Queries on Knowledge Graphscited by 21
  3. 2022Faithful to the Document or to the World? Mitigating Hallucinations via Entity-linked Knowledge in Abstractive Summarizationcited by 20
  4. 2021Reinforcement Learning from Reformulations in Conversational Question Answering over Knowledge Graphscited by 18
  5. 2022Reinforcement learning on graphs: A surveycited by 18
  6. 2022Few-shot Relational Reasoning via Connection Subgraph Pretrainingcited by 18
  7. 2024LLM-Based Multi-Hop Question Answering with Knowledge Graph Integration in Evolving Environmentscited by 18
  8. 2021Towards Accurate and Compact Architectures via Neural Architecture Transformercited by 17
  9. 2021Learning to Walk with Dual Agents for Knowledge Graph Reasoningcited by 16
  10. 2021Efficient Non-Sampling Knowledge Graph Embeddingcited by 14
  11. 2024Structure Guided Prompt: Instructing Large Language Model in Multi-Step Reasoning by Exploring Graph Structure of the Textcited by 14
  12. 2022CAKE: A Scalable Commonsense-Aware Framework For Multi-View Knowledge Graph Completioncited by 13
  13. 2022MMKGR: Multi-hop Multi-modal Knowledge Graph Reasoningcited by 13
  14. 2022SMiLE: Schema-augmented Multi-level Contrastive Learning for Knowledge Graph Link Predictioncited by 12
  15. 2024OCEAN: Offline Chain-of-thought Evaluation and Alignment in Large Language Modelscited by 12
  16. 2021Is Multi-Hop Reasoning Really Explainable? Towards Benchmarking Reasoning Interpretabilitycited by 10
  17. 2022KRACL: Contrastive Learning with Graph Context Modeling for Sparse Knowledge Graph Completioncited by 10
  18. 2021Are Missing Links Predictable? An Inferential Benchmark for Knowledge Graph Completioncited by 9
  19. 2021Knowledge Sheaves: A Sheaf-Theoretic Framework for Knowledge Graph Embeddingcited by 9
  20. 2022Great Truths are Always Simple: A Rather Simple Knowledge Encoder for Enhancing the Commonsense Reasoning Capacity of Pre-Trained Modelscited by 9
  21. 2023Learning Representations of Bi-level Knowledge Graphs for Reasoning beyond Link Predictioncited by 9
  22. 2023SARF: Aliasing Relation Assisted Self-Supervised Learning for Few-shot Relation Reasoningcited by 9
  23. 2023Logical Entity Representation in Knowledge-Graphs for Differentiable Rule Learningcited by 8
  24. 2024Comprehensible Artificial Intelligence on Knowledge Graphs: A surveycited by 8
  25. 2021Combining Rules and Embeddings via Neuro-Symbolic AI for Knowledge Base Completioncited by 7
  26. 2021Explainable Biomedical Recommendations via Reinforcement Learning Reasoning on Knowledge Graphscited by 7
  27. 2022R5: Rule Discovery with Reinforced and Recurrent Relational Reasoningcited by 7
  28. 2022RLET: A Reinforcement Learning Based Approach for Explainable QA with Entailment Treescited by 7
  29. 2026Knowledge Graphs are Implicit Reward Models: Path-Derived Signals Enable Compositional Reasoningcited by 7
  30. 2021Perform Like an Engine: A Closed-Loop Neural-Symbolic Learning Framework for Knowledge Graph Inferencecited by 6
  31. 2022SQUIRE: A Sequence-to-sequence Framework for Multi-hop Knowledge Graph Reasoningcited by 6
  32. 2022Inductive Knowledge Graph Reasoning for Multi-batch Emerging Entitiescited by 6
  33. 2023A Survey On Few-shot Knowledge Graph Completion with Structural and Commonsense Knowledgecited by 6
  34. 2021Coarse-grained decomposition and fine-grained interaction for multi-hop question answeringcited by 5
  35. 2023Causal Question Answering with Reinforcement Learningcited by 5
  36. 2024Generating SROI^- Ontologies via Knowledge Graph Query Embedding Learningcited by 5
  37. 2021Rule Induction in Knowledge Graphs Using Linear Programmingcited by 4
  38. 2022KGML-xDTD: A Knowledge Graph-based Machine Learning Framework for Drug Treatment Prediction and Mechanism Descriptioncited by 4

50 most cited of 102 All 102 (JSON)