mlsanityData to 14 Oct 2026

Critic-R: Improving Agentic Search using Instruction-tuned Retrievers with Natural Language Introspective Feedback

arXiv 2606.00590 ↗First posted 30 May 2026cs.IR cs.AIcited by 0

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Agentic search systems iteratively interact with retrieval models to answer complex queries. Despite substantial progress, optimizing retrievers for agentic search remains challenging, often requiring heavy co-training or gold-standard annotations that limit real-world applicability. We propose Critic-R, a framework that explicitly closes the feedback loop between the reasoning agent and the retrieval model during both inference and training. Critic-R introduces a critic model that evaluates the agent's introspective reasoning trace after consuming retrieved evidence to determine whether the retrieved context sufficiently supports the next reasoning step. Critic-R has two complementary mechanisms: Critic-R-Zero, an inference-time query refinement loop that iteratively rewrites queries and retrieval instructions, and Critic-Embed, an optimization approach for retrieval models that leverages successful and failed refinement trajectories as automatic supervision without requiring manual relevance annotation. We evaluate Critic-R on HotpotQA, 2WikiMultihopQA, MuSiQue, and Bamboogle. Results show that Critic-R significantly improves both retrieval quality and downstream answer accuracy.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 22 of its 32 references that are in the map. First posted on arXiv 30 May 2026.

Builds on

22 of 32 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026CoSearch: Joint Training of Reasoning and Document Ranking via Reinforcement Learning for Agentic Search19 Apr
  2. 2026Agentic-R: Learning to Retrieve for Agentic Search17 Jan

Other references · 20

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  5. 2023Let's Verify Step by Stepcited by 3,789
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  7. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  8. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  9. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  10. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  11. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  12. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
8 more
  1. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  2. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
  3. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
  4. 2023In-Context Retrieval-Augmented Language Modelscited by 702
  5. 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 346
  6. 2024Jasper and Stella: distillation of SOTA embedding modelscited by 112
  7. 2024Stochastic RAG: End-to-End Retrieval-Augmented Generation through Expected Utility Maximizationcited by 29
  8. 2025Training a Utility-based Retriever Through Shared Context Attribution for Retrieval-Augmented Language Modelscited by 13

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Rethinking Reasoning-Intensive Retrieval: Evaluating and Advancing Retrievers in Agentic Search Systems
  2. 2026RICE-PO: Turning Retrieval Interactions into Credit Signals for Reasoning Agents
  3. 2026R2-Searcher: Calibrating Retrieval and Reasoning Boundaries for Agentic Search
  4. 2026AgentIR: Reasoning-Aware Retrieval for Deep Research Agents
  5. 2026GRASP: GRanularity-Aware Search Policy for Agentic RAG
  6. 2026Optimizing Agentic Reasoning with Retrieval via Synthetic Semantic Information Gain Reward
  7. 2026Compute Allocation for Reasoning-Intensive Retrieval Agents
  8. 2026Beyond Semantic Similarity: Performance and Costs of Agentic Retrieval for Complex Tasks