mlsanityData to 14 Oct 2026

RAG-Star: Enhancing Deliberative Reasoning with Retrieval Augmented Verification and Refinement

arXiv 2412.12881 ↗First posted 17 Dec 2024cs.CL cs.AIcited by 40

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

also in 1 more
Abstract (arXiv)

Existing large language models (LLMs) show exceptional problem-solving capabilities but might struggle with complex reasoning tasks. Despite the successes of chain-of-thought and tree-based search methods, they mainly depend on the internal knowledge of LLMs to search over intermediate reasoning steps, limited to dealing with simple tasks involving fewer reasoning steps. In this paper, we propose RAG-Star, a novel RAG approach that integrates the retrieved information to guide the tree-based deliberative reasoning process that relies on the inherent knowledge of LLMs. By leveraging Monte Carlo Tree Search, RAG-Star iteratively plans intermediate sub-queries and answers for reasoning based on the LLM itself. To consolidate internal and external knowledge, we propose an retrieval-augmented verification that utilizes query- and answer-aware reward modeling to provide feedback for the inherent reasoning of LLMs. Our experiments involving Llama-3.1-8B-Instruct and GPT-4o demonstrate that RAG-Star significantly outperforms previous RAG and reasoning methods.

Key facts. Cited by 40 papers in the map (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 25 of its 39 references that are in the map. First posted on arXiv 17 Dec 2024.

Builds on

25 of 39 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Survey18 Dec
  2. 2023Search-in-the-Chain: Interactively Enhancing Large Language Models with Search for Knowledge-intensive Tasks28 Apr

Other references · 23

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  5. 2023Let's Verify Step by Stepcited by 3,789
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  7. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  8. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  9. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  10. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  11. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  12. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
11 more
  1. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  2. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
  3. 2022Towards Reasoning in Large Language Models: A Surveycited by 596
  4. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 301
  5. 2024Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoningcited by 279
  6. 2023Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Frameworkcited by 139
  7. 2024Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planningcited by 97
  8. 2024LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoningcited by 91
  9. 2022Solving Math Word Problems via Cooperative Reasoning induced Language Modelscited by 77
  10. 2024OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AIcited by 71
  11. 2019Multi-hop Question Answering via Reasoning Chainscited by 42

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 27; 2026: 11.

40 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  2. 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 336
  3. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 335
  4. 2025ZeroSearch: Incentivize the Search Capability of LLMs without Searchingcited by 208
  5. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 111
  6. 2025SimpleDeepSearcher: Deep Information Seeking via Web-Powered Reasoning Trajectory Synthesiscited by 55
  7. 2025Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMscited by 42
  8. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  9. 2025Improving Retrieval-Augmented Generation through Multi-Agent Reinforcement Learningcited by 39
  10. 2025MAS-ZERO: Designing Multi-Agent Systems with Zero Supervisioncited by 28
  11. 2025TIGeR: Tool-Integrated Geometric Reasoning in Vision-Language Models for Roboticscited by 27
  12. 2025A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applicationscited by 22
28 more
  1. 2025MCTS-RAG: Enhancing Retrieval-Augmented Generation with Monte Carlo Tree Searchcited by 21
  2. 2025Collab-RAG: Boosting Retrieval-Augmented Generation for Complex Question Answering via White-Box and Black-Box LLM Collaborationcited by 20
  3. 2025Towards AI Search Paradigmcited by 19
  4. 2026SmartSearch: Process Reward-Guided Query Refinement for Search Agentscited by 14
  5. 2025ReaRAG: Knowledge-guided Reasoning Enhances Factuality of Large Reasoning Models with Iterative Retrieval Augmented Generationcited by 13
  6. 2025AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforced Self-Playcited by 10
  7. 2025DeepSolution: Boosting Complex Engineering Solution Design via Tree-based Exploration and Bi-point Thinkingcited by 9
  8. 2025DecEx-RAG: Boosting Agentic Retrieval-Augmented Generation with Decision and Execution Optimization via Process Supervisioncited by 9
  9. 2026Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluationcited by 7
  10. 2025Uncertainty Quantification for Retrieval-Augmented Reasoningcited by 6
  11. 2025ARise: Towards Knowledge-Augmented Reasoning via Risk-Adaptive Searchcited by 5
  12. 2025PIS: Linking Importance Sampling and Attention Mechanisms for Efficient Prompt Compressioncited by 4
  13. 2025CAFE: Retrieval Head-based Coarse-to-Fine Information Seeking to Enhance Multi-Document QA Capabilitycited by 4
  14. 2025Reason to Retrieve: Enhancing Query Understanding through Decomposition and Interpretationcited by 4
  15. 2026PiCA: Pivot-Based Credit Assignment for Search Agentic Reinforcement Learningcited by 3
  16. 2025Self-Critique Guided Iterative Reasoning for Multi-hop Question Answeringcited by 2
  17. 2025MARS: Co-evolving Dual-System Deep Research via Multi-Agent Reinforcement Learningcited by 2
  18. 2026DuMate-DeepResearch: An Auditable Multi-Agent System with Recursive Search and Rubric-Grounded Reasoningcited by 2
  19. 2026Track, Rank, Crack: Epistemic Working Memory Scales Multi-Hop Reasoning in Language Agentscited by 2
  20. 2024A Survey of Query Optimization in Large Language Modelscited by 1
  21. 2026Anchored Confabulation: Partial Evidence Non-Monotonically Amplifies Confident Hallucination in LLMscited by 1
  22. 2024An Overview and Discussion on Using Large Language Models for Implementation Generation of Solutions to Open-Ended Problemscited by 0
  23. 2025Say Less, Mean More: Leveraging Pragmatics in Retrieval-Augmented Generationcited by 0
  24. 2026Quantifying Prior Dominance in RAG Systemscited by 0
  25. 2026STEC: Evidence Compression for Deep Search in Open-domain Multi-Hop QAcited by 0
  26. 2026GRADRAG: Cross-Component Prompt Adaptation for Coordinated Multi-Agent RAGcited by 0
  27. 2026From Found to Designed: Concepts as a Design Axis for Large Language Modelscited by 0
  28. 2026When LLM Meets Tree Search: A Systematic View of Inference as Search in Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025From Sufficiency to Reflection: Reinforcement-Guided Thinking Quality in Retrieval-Augmented Reasoning for LLMs
  2. 2025From Facts to Conclusions : Integrating Deductive Reasoning in Retrieval-Augmented LLMs
  3. 2025Synergizing RAG and Reasoning: A Systematic Review
  4. 2025Reasoning RAG via System 1 or System 2: A Survey on Reasoning Agentic Retrieval-Augmented Generation for Industry Challenges
  5. 2025You Don't Need Pre-built Graphs for RAG: Retrieval Augmented Generation with Adaptive Reasoning Structures
  6. 2026Self-Correcting RAG: Enhancing Faithfulness via MMKP Context Selection and NLI-Guided MCTS
  7. 2025RAG-R1: Incentivizing the Search and Reasoning Capabilities of LLMs through Multi-query Parallelism
  8. 2025UR2: Unify RAG and Reasoning through Reinforcement Learning