# Search-P1: Path-Centric Reward Shaping for Stable and Efficient Agentic RAG Training

arXiv: https://arxiv.org/abs/2602.22576 · HTML: https://mlsanity.com/paper/2602.22576/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Agentic RAG, agents and memory, one of 15 directions in Retrieval-augmented generation.
- Builds on 19 of its 29 references that are in the map.
- First posted on arXiv 26 Feb 2026.

## Placed in

- Retrieval-augmented generation / [Agentic RAG, agents and memory](https://mlsanity.com/rag/agentic-rag-agents-and-memory/)
- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 19 of 19)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://mlsanity.com/paper/2503.09516/) (2025)
- [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://mlsanity.com/paper/2011.01060/) (2020)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [Measuring and Narrowing the Compositionality Gap in Language Models](https://mlsanity.com/paper/2210.03350/) (2022)
- [Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions](https://mlsanity.com/paper/2212.10509/) (2022)
- [Search-o1: Agentic Search-Enhanced Large Reasoning Models](https://mlsanity.com/paper/2501.05366/) (2025)
- [Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG](https://mlsanity.com/paper/2501.09136/) (2025)
- [Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT](https://mlsanity.com/paper/2302.10198/) (2023)
- [Process vs. Outcome Reward: Which is Better for Agentic RAG Reinforcement Learning](https://mlsanity.com/paper/2505.14069/) (2025)
- [ReARTeR: Retrieval-Augmented Reasoning with Trustworthy Process Rewarding](https://mlsanity.com/paper/2501.07861/) (2025)
- [RAG-RL: Advancing Retrieval-Augmented Generation via RL and Curriculum Learning](https://mlsanity.com/paper/2503.12759/) (2025)
- [Search Wisely: Mitigating Sub-optimal Agentic Searches By Reducing Uncertainty](https://mlsanity.com/paper/2505.17281/) (2025)
- [Improving Complex Reasoning over Knowledge Graph with Logic-Aware Curriculum Tuning](https://mlsanity.com/paper/2405.01649/) (2024)

## Built on by (most cited first, 18 of 18)

- [GraSP: Graph-Structured Skill Compositions for LLM Agents](https://mlsanity.com/paper/2604.17870/) (2026)
- [Search-E1: Self-Distillation Drives Self-Evolution in Search-Augmented Reasoning](https://mlsanity.com/paper/2605.22511/) (2026)
- [OASES: Outcome-Aligned Search-Evaluation Co-Training for Agentic Search](https://mlsanity.com/paper/2604.03675/) (2026)
- [ARBOR: Online Process Rewards via a Reusable Rubric Buffer for Search Agents](https://mlsanity.com/paper/2606.03239/) (2026)
- [SearchArt: Training Long-Horizon Search Agent with Scalable Synthetic and Verified Task](https://mlsanity.com/paper/2607.24850/) (2026)
- [What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agents](https://mlsanity.com/paper/2608.27260/) (2026)
- [Knowledge-Graph Paths as Intermediate Supervision for Self-Evolving Search Agents](https://mlsanity.com/paper/2605.05702/) (2026)
- [From Verdict to Process: Agentic Reinforcement Learning for Multi-Stage Fact Verification](https://mlsanity.com/paper/2606.13262/) (2026)
- [RAG-Stress: Probing the Limits of Evidence Reliance in Retrieval-Augmented Generation](https://mlsanity.com/paper/2610.11183/) (2026)
- [Co-Evolving Graph and Text Memory for Training-Free Multi-Hop Question Answering](https://mlsanity.com/paper/2607.23278/) (2026)
- [HindSearch: Trajectory-Level Hindsight Critique for Search-Augmented Reinforcement Learning](https://mlsanity.com/paper/2608.01597/) (2026)
- [MetaRAG: Belief-Action Aligned Policy Optimization for Agentic RAG](https://mlsanity.com/paper/2608.24214/) (2026)
- [Distillation as Probability Transport: Routed On-Policy Distillation](https://mlsanity.com/paper/2609.08337/) (2026)
- [Reinforcement Learning with Verifiable Rewards for Small Search Agents](https://mlsanity.com/paper/2609.28765/) (2026)
- [PrimeSeeker: Capability-Oriented Supervision for Deep Search Agents](https://mlsanity.com/paper/2609.35816/) (2026)
- [LexiHorizon: Stabilizing Reinforcement Learning for Long-Horizon Deep Search](https://mlsanity.com/paper/2610.05119/) (2026)
- [Neos: Learning to Interleave Search and Visual Reasoning](https://mlsanity.com/paper/2610.17043/) (2026)
- [Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewards](https://mlsanity.com/paper/2610.18868/) (2026)

## Abstract (arXiv)

Retrieval-Augmented Generation (RAG) enhances large language models (LLMs) by incorporating external knowledge, yet traditional single-round retrieval struggles with complex multi-step reasoning. Agentic RAG addresses this by enabling LLMs to dynamically decide when and what to retrieve, but current RL-based training methods suffer from sparse outcome rewards that discard intermediate signals and low sample efficiency where failed samples contribute nothing. We propose Search-P1, a framework that introduces path-centric reward shaping for agentic RAG training, comprising two key components: (1) Path-Centric Reward, which evaluates the structural quality of reasoning trajectories through order-agnostic step coverage and soft scoring that extracts learning signals even from failed samples, and (2) Dual-Track Path Scoring with offline-generated reference planners that assesses paths from both self-consistency and reference-alignment perspectives. Experiments on multiple QA benchmarks demonstrate that Search-P1 achieves significant improvements over Search-R1 and other strong baselines, with an average accuracy gain of 7.7 points.