mlsanityData to 7 Oct 2026

DeepRefine: Agentic Knowledge Refinement via Reinforcement Learning

arXiv 2605.10488 ↗First posted 11 May 2026cs.CL cs.AIcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

External knowledge enables large language model (LLM) agents to ground their actions and decisions beyond intrinsic parametric memory in open-ended, knowledge-intensive downstream tasks. Yet the quality of the underlying knowledge bases is systematically limited by incompleteness, incorrectness, or redundancy, manifested as missing evidence or cross-document links, low-confidence or imprecise claims, and ambiguous or coreference resolution issues. Such defects compound under iterative use, degrading retrieval fidelity and downstream task performance. We present DeepRefine, a reinforcement learning framework for agentic knowledge refinement that evolves the quality of any pre-constructed structured knowledge bases, e.g., knowledge graphs or LLM-Wikis, with user queries to make it more suitable for the downstream tasks. DeepRefine performs multi-turn interactions with the knowledge base and conducts abductive diagnosis over the interaction history, localizes likely defects, and executes targeted refinement actions for incremental knowledge base updates. To further optimize refinement policies of DeepRefine without golden refinement trajectories, we introduce a Gain-Beyond-Draft (GBD) reward and train the reasoning process end-to-end via reinforcement learning. Extensive experiments demonstrate consistent downstream gains over strong baselines.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 33 of its 46 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

33 of 46 references.

References · 33

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2025Qwen3 Technical Reportcited by 8,195
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  7. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  8. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  9. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
  10. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  11. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  12. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
21 more
  1. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  2. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  3. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 800
  4. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 722
  5. 2025Zep: A Temporal Knowledge Graph Architecture for Agent Memorycited by 372
  6. 2019KG-BERT: BERT for Knowledge Graph Completioncited by 323
  7. 2024LightRAG: Simple and Fast Retrieval-Augmented Generationcited by 310
  8. 2024HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Modelscited by 273
  9. 2025From RAG to Memory: Non-Parametric Continual Learning for Large Language Modelscited by 223
  10. 2025Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learningcited by 220
  11. 2021MasakhaNER: Named Entity Recognition for African Languagescited by 155
  12. 2025A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challengescited by 58
  13. 2025Graph-R1: Towards Agentic GraphRAG Framework via End-to-end Reinforcement Learningcited by 50
  14. 2025Retrieval-Augmented Generation with Hierarchical Knowledgecited by 46
  15. 2025s3: You Don't Need That Much Data to Train a Search Agent via RLcited by 41
  16. 2026Graph-based Agent Memory: Taxonomy, Techniques, and Applicationscited by 22
  17. 2025LeanRAG: Knowledge-Graph-Based Generation with Semantic Aggregation and Hierarchical Retrievalcited by 14
  18. 2024Neuro-Symbolic Entity Alignment via Variational Inferencecited by 8
  19. 2025AutoGraph-R1: End-to-End Reinforcement Learning for Knowledge Graph Constructioncited by 6
  20. 2021KGRefiner: Knowledge Graph Refinement for Improving Accuracy of Translational Link Prediction Methodscited by 1
  21. 2025TRAIL: Joint Inference and Refinement of Knowledge Graphs with Large Language Modelscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Search and Refine During Think: Facilitating Knowledge Refinement for Improved Retrieval-Augmented Reasoning
  2. 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skills
  3. 2025AgentRefine: Enhancing Agent Generalization through Refinement Tuning
  4. 2026Learning Agentic Policy from Action Guidance
  5. 2026Subagents vs Agent Skills: Executing Reusable Knowledge for Long-Horizon Agentic Tasks
  6. 2026Forgotten in Weights, Recovered by Tools: Agentic Tool Unlearning for LLM Agents
  7. 2026Efficient Agentic Reinforcement Learning with On-Policy Intrinsic Knowledge Boundary Enhancement
  8. 2026NLPG: Natural-Language Policy Gradients for Self-Evolving Language Agents