mlsanityData to 7 Oct 2026

SoK: Agentic Retrieval-Augmented Generation (RAG): Taxonomy, Architectures, Evaluation, and Research Directions

arXiv 2603.07379 ↗First posted 7 Mar 2026cs.AI cs.CL cs.CRcited by 7

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

also in 2 more
Abstract (arXiv)

Retrieval-Augmented Generation (RAG) systems are increasingly evolving into agentic architectures where large language models autonomously coordinate multi-step reasoning, dynamic memory management, and iterative retrieval strategies. Despite rapid industrial adoption, current research lacks a systematic understanding of Agentic RAG as a sequential decision-making system, leading to highly fragmented architectures, inconsistent evaluation methodologies, and unresolved reliability risks. This Systematization of Knowledge (SoK) paper provides the first unified framework for understanding these autonomous systems. We formalize agentic retrieval-generation loops as finite-horizon partially observable Markov decision processes, explicitly modeling their control policies and state transitions. Building upon this formalization, we develop a comprehensive taxonomy and modular architectural decomposition that categorizes systems by their planning mechanisms, retrieval orchestration, memory paradigms, and tool-invocation behaviors. We further analyze the critical limitations of traditional static evaluation practices and identify severe systemic risks inherent to autonomous loops, including compounding hallucination propagation, memory poisoning, retrieval misalignment, and cascading tool-execution vulnerabilities. Finally, we outline key doctoral-scale research directions spanning stable adaptive retrieval, cost-aware orchestration, formal trajectory evaluation, and oversight mechanisms, providing a definitive roadmap for building reliable, controllable, and scalable agentic retrieval systems.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 83 of its 105 references that are in the map. First posted on arXiv 7 Mar 2026.

Builds on

83 of 105 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026A-RAG: Scaling Agentic Retrieval-Augmented Generation via Hierarchical Retrieval Interfaces3 Feb
  2. 2026The P-Completeness of Inverted Index Traversal: On the Complexity of Evaluating Boolean Query DAGs26 Jan
  3. 2025MA-RAG: Multi-Agent Retrieval-Augmented Generation via Collaborative Chain-of-Thought Reasoning26 May
  4. 2023Retrieval-Augmented Generation for Large Language Models: A Survey18 Dec
  5. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions20 Dec

Other references · 78

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  7. 2020Scaling Laws for Neural Language Modelscited by 6,411
  8. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  9. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  10. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  11. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  12. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
38 more
  1. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  2. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  3. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  4. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  5. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  6. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  7. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  8. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  9. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  10. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  11. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  12. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  13. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  14. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  15. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  16. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  17. 2023ChatDev: Communicative Agents for Software Developmentcited by 893
  18. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  19. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  20. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 550
  21. 2023Active Retrieval Augmented Generationcited by 506
  22. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
  23. 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 343
  24. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 326
  25. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 319
  26. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 317
  27. 2023Benchmarking Large Language Models in Retrieval-Augmented Generationcited by 307
  28. 2022Teaching language models to support answers with verified quotescited by 290
  29. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  30. 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 251
  31. 2025Agentic Retrieval-Augmented Generation: A Survey on Agentic RAGcited by 242
  32. 2023Query Rewriting for Retrieval-Augmented Large Language Modelscited by 211
  33. 2025Plan-and-Act: Improving Planning of Agents for Long-Horizon Taskscited by 190
  34. 2025Survey on Evaluation of LLM-based Agentscited by 190
  35. 2023ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Modelscited by 174
  36. 2024Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Toolscited by 170
  37. 2022MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoningcited by 153
  38. 2025Evaluation and Benchmarking of LLM Agents: A Surveycited by 145

50 most cited of 78 All 78 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineeringcited by 68
  2. 2026PYTHALAB-MERA: Validation-Grounded Memory, Retrieval, and Acceptance Control for Frozen-LLM Coding Agentscited by 0
  3. 2026Cascading Hallucination in Agentic RAG: The CHARM Framework for Detection and Mitigationcited by 0
  4. 2026Agent-Orchestrated Adaptive RAG: A Comparative Study on Structured and Multi-Hop Retrievalcited by 0
  5. 2026LayerRAG-Bench: A Cross-Layer Reliability Benchmark for Agentic Retrieval-Augmented Generationcited by 0
  6. 2026OpenClaw and Ollama in Agentic AI: Toward Fully Autonomous and Scalable AI Agent Systemscited by 0
  7. 2026Symbolic Separation: Grounding Deep Agents in Knowledge Graphs for Trustworthy Operational Data Analyticscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Reasoning RAG via System 1 or System 2: A Survey on Reasoning Agentic Retrieval-Augmented Generation for Industry Challenges
  2. 2025RAGCap-Bench: Benchmarking Capabilities of LLMs in Agentic Retrieval Augmented Generation Systems
  3. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning
  4. 2025A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applications
  5. 2025DecEx-RAG: Boosting Agentic Retrieval-Augmented Generation with Decision and Execution Optimization via Process Supervision
  6. 2026GRASP: GRanularity-Aware Search Policy for Agentic RAG
  7. 2025Interact-RAG: Reason and Interact with the Corpus, Beyond Black-Box Retrieval
  8. 2026Rethinking Agentic RAG: Toward LLM-Driven Logical Retrieval Beyond Embeddings