# Reinforcement Learning Enhanced Multi-hop Reasoning for Temporal Knowledge Question Answering

arXiv: https://arxiv.org/abs/2601.01195 · HTML: https://mlsanity.com/paper/2601.01195/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 26 of its 71 references that are in the map.
- First posted on arXiv 3 Jan 2026.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)

## Builds on (most cited first, 26 of 26)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [Group Sequence Policy Optimization](https://mlsanity.com/paper/2507.18071/) (2025)
- [Extending Context Window of Large Language Models via Positional Interpolation](https://mlsanity.com/paper/2306.15595/) (2023)
- [Inference-Time Scaling for Generalist Reward Modeling](https://mlsanity.com/paper/2504.02495/) (2025)
- [Reasoning Models Can Be Effective Without Thinking](https://mlsanity.com/paper/2504.09858/) (2025)
- [RM-R1: Reward Modeling as Reasoning](https://mlsanity.com/paper/2505.02387/) (2025)
- [Med-R1: Reinforcement Learning for Generalizable Medical Reasoning in Vision-Language Models](https://mlsanity.com/paper/2503.13939/) (2025)
- [Thinkless: LLM Learns When to Think](https://mlsanity.com/paper/2505.13379/) (2025)
- [Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learning](https://mlsanity.com/paper/2503.16252/) (2025)
- [Think Only When You Need with Large Hybrid-Reasoning Models](https://mlsanity.com/paper/2505.14631/) (2025)
- [DianJin-R1: Evaluating and Enhancing Financial Reasoning in Large Language Models](https://mlsanity.com/paper/2504.15716/) (2025)
- [DABERT: Dual Attention Enhanced BERT for Semantic Matching](https://mlsanity.com/paper/2210.03454/) (2022)
- [Improving Semantic Matching through Dependency-Enhanced Pre-trained Model with Adaptive Fusion](https://mlsanity.com/paper/2210.08471/) (2022)
- [Searching for Optimal Subword Tokenization in Cross-domain NER](https://mlsanity.com/paper/2206.03352/) (2022)
- [Robust Lottery Tickets for Pre-trained Language Models](https://mlsanity.com/paper/2211.03013/) (2022)
- [Who Stole Your Data? A Method for Detecting Unauthorized RAG Theft](https://mlsanity.com/paper/2510.07728/) (2025)
- [Progressive Mastery: Customized Curriculum Learning with Guided Prompting for Mathematical Reasoning](https://mlsanity.com/paper/2506.04065/) (2025)
- [Not All Parameters Are Created Equal: Smart Isolation Boosts Fine-Tuning Performance](https://mlsanity.com/paper/2508.21741/) (2025)
- [Resolving Word Vagueness with Scenario-guided Adapter for Natural Language Inference](https://mlsanity.com/paper/2405.12434/) (2024)
- [HoPE: Hyperbolic Rotary Positional Encoding for Stable Long-Range Dependency Modeling in Large Language Models](https://mlsanity.com/paper/2509.05218/) (2025)
- [Comateformer: Combined Attention Transformer for Semantic Sentence Matching](https://mlsanity.com/paper/2412.07220/) (2024)

## Built on by (most cited first, 2 of 2)

- [Why Supervised Fine-Tuning Fails to Learn: A Systematic Study of Incomplete Learning in Large Language Models](https://mlsanity.com/paper/2604.10079/) (2026)
- [Reason Only When Needed: Efficient Generative Reward Modeling via Model-Internal Uncertainty](https://mlsanity.com/paper/2604.10072/) (2026)

## Abstract (arXiv)

Temporal knowledge graph question answering (TKGQA) involves multi-hop reasoning over temporally constrained entity relationships in the knowledge graph to answer a given question. However, at each hop, large language models (LLMs) retrieve subgraphs with numerous temporally similar and semantically complex relations, increasing the risk of suboptimal decisions and error propagation. To address these challenges, we propose the multi-hop reasoning enhanced (MRE) framework, which enhances both forward and backward reasoning to improve the identification of globally optimal reasoning trajectories. Specifically, MRE begins with prompt engineering to guide the LLM in generating diverse reasoning trajectories for a given question. Valid reasoning trajectories are then selected for supervised fine-tuning, serving as a cold-start strategy. Finally, we introduce Tree-Group Relative Policy Optimization (T-GRPO), a recursive, tree-structured learning-by-exploration approach. At each hop, exploration establishes strong causal dependencies on the previous hop, while evaluation is informed by multi-path exploration feedback from subsequent hops. Experimental results on two TKGQA benchmarks indicate that the proposed MRE-based model consistently surpasses state-of-the-art (SOTA) approaches in handling complex multi-hop queries. Further analysis highlights improved interpretability and robustness to noisy temporal annotations.