# A-RAG: Scaling Agentic Retrieval-Augmented Generation via Hierarchical Retrieval Interfaces

arXiv: https://arxiv.org/abs/2602.03442 · HTML: https://mlsanity.com/paper/2602.03442/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 16 of its 32 references that are in the map.
- First posted on arXiv 3 Feb 2026.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)
- Retrieval-augmented generation / [Agentic RAG, agents and memory](https://mlsanity.com/rag/agentic-rag-agents-and-memory/)
- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 16 of 16)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](https://mlsanity.com/paper/2404.16130/) (2024)
- [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://mlsanity.com/paper/2503.09516/) (2025)
- [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://mlsanity.com/paper/2011.01060/) (2020)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions](https://mlsanity.com/paper/2212.10509/) (2022)
- [MuSiQue: Multihop Questions via Single-hop Question Composition](https://mlsanity.com/paper/2108.00573/) (2021)
- [RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval](https://mlsanity.com/paper/2401.18059/) (2024)
- [Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity](https://mlsanity.com/paper/2403.14403/) (2024)
- [MemoRAG: Boosting Long Context Processing with Global Memory-Enhanced Retrieval Augmentation](https://mlsanity.com/paper/2409.05591/) (2024)
- [DeepSeekMath-V2: Towards Self-Verifiable Mathematical Reasoning](https://mlsanity.com/paper/2511.22570/) (2025)
- [LinearRAG: Linear Graph Retrieval Augmented Generation on Large-scale Corpora](https://mlsanity.com/paper/2510.10114/) (2025)
- [RA-ISF: Learning to Answer and Understand from Retrieval Augmentation via Iterative Self-Feedback](https://mlsanity.com/paper/2403.06840/) (2024)

## Built on by (most cited first, 24 of 24)

- [SoK: Agentic Retrieval-Augmented Generation (RAG): Taxonomy, Architectures, Evaluation, and Research Directions](https://mlsanity.com/paper/2603.07379/) (2026)
- [Corpus2Skill: Distilling Enterprise Knowledge into Navigable Agent Skills for QA and RAG](https://mlsanity.com/paper/2604.14572/) (2026)
- [Token Economics for LLM Agents: A Dual-View Study from Computing and Economics](https://mlsanity.com/paper/2605.09104/) (2026)
- [Rethinking Agentic RAG: Toward LLM-Driven Logical Retrieval Beyond Embeddings](https://mlsanity.com/paper/2605.27123/) (2026)
- [An Empirical Study of Proactive Coding Assistants in Real-World Software Development](https://mlsanity.com/paper/2605.05700/) (2026)
- [Query Symbolically or Retrieve Semantically? A Dataset and Method for Semi-Structured Question Answering](https://mlsanity.com/paper/2605.27164/) (2026)
- [Better Later Than Sooner: Neuro-Symbolic Knowledge Graph Construction via Ontology-grounded Post-extraction Correction](https://mlsanity.com/paper/2605.29168/) (2026)
- [Beyond the Reranker: Do RAG Retrieval Enhancements Help Once a Strong Reranker Is Present?](https://mlsanity.com/paper/2606.28367/) (2026)
- [GRASP: GRanularity-Aware Search Policy for Agentic RAG](https://mlsanity.com/paper/2607.10463/) (2026)
- [Keep It InMind: Benchmarking the Implicit-Association Blind Spot in Agent Memory](https://mlsanity.com/paper/2607.24368/) (2026)
- [CoSA: Accelerating Long-Context Inference via Proxy-Kernel Co-Designed Sparse Attention](https://mlsanity.com/paper/2607.25291/) (2026)
- [When Your Agent Opens the Chat App: Agent-Controlled Search over Raw Chat Logs Rivals Structured Memory](https://mlsanity.com/paper/2608.12888/) (2026)
- [StateRAG: Typed State Contracts for Complex Retrieval-Augmented Generation](https://mlsanity.com/paper/2605.25379/) (2026)
- [Narrative Knowledge Weaver: Narrative-Centric Retrieval-Augmented Reasoning for Long-Form Text Understanding](https://mlsanity.com/paper/2606.05724/) (2026)
- [MCompassRAG: Topic Metadata as a Semantic Compass for Paragraph-Level Retrieval](https://mlsanity.com/paper/2606.18508/) (2026)
- [Constraint-Aware Hierarchical Search for Regulation-Driven Fine-Grained Classification](https://mlsanity.com/paper/2607.10588/) (2026)
- [Towards Trustworthy and Cost-Efficient Data Integration: From Na"ive RAG to Agentic RAG](https://mlsanity.com/paper/2607.22319/) (2026)
- [ScalableRAG: High-Quality RAG at Zero Ingestion Cost](https://mlsanity.com/paper/2607.25135/) (2026)
- [DocNavRAG: Document-Structured Graph RAG with Stateful Evidence Construction for Complex Document Question Answering](https://mlsanity.com/paper/2608.01565/) (2026)
- [Before Reasoning Can Fail: Pre-Evidence Procedural Failures in Agentic RAG](https://mlsanity.com/paper/2608.02011/) (2026)
- [VDGR-RAG: Vectors, Directories, Graphs, and Reflection Are All You Need for Unified Reasoning over Hierarchical Enterprise Knowledge](https://mlsanity.com/paper/2608.07994/) (2026)
- [SearchWiki: Learning to Build and Navigate Knowledge Wikis for Active Information Seeking](https://mlsanity.com/paper/2608.29953/) (2026)
- [TRACE: Accountable Agentic Retrieval for Source Discovery in Digital Archives](https://mlsanity.com/paper/2609.19897/) (2026)
- [Realize What Matters: Principled Context Representation for Large-Scale Reasoning](https://mlsanity.com/paper/2609.27173/) (2026)

## Abstract (arXiv)

Frontier language models have demonstrated strong reasoning and long-horizon tool-use capabilities. However, existing RAG systems fail to leverage these capabilities. They still rely on two paradigms: (1) designing an algorithm that retrieves passages in a single shot and concatenates them into the model's input, or (2) predefining a workflow and prompting the model to execute it step-by-step. Neither paradigm allows the model to participate in retrieval decisions, preventing efficient scaling with model improvements. In this paper, we introduce A-RAG, an Agentic RAG framework that exposes hierarchical retrieval interfaces directly to the model. A-RAG provides three retrieval tools: keyword search, semantic search, and chunk read, enabling the agent to adaptively search and retrieve information across multiple granularities. Experiments on multiple open-domain QA benchmarks show that A-RAG consistently outperforms existing approaches with comparable or lower retrieved tokens, demonstrating that A-RAG effectively leverages model capabilities and dynamically adapts to different RAG tasks. We further systematically study how A-RAG scales with model size and test-time compute. We will release our code and evaluation suite to facilitate future research. Code and evaluation suite are available at https://github.com/Ayanami0730/arag.