mlsanityData to 14 Oct 2026

Atom-Searcher: Enhancing Agentic Deep Research via Fine-Grained Atomic Thought Reward

arXiv 2508.12800 ↗First posted 18 Aug 2025cs.CL cs.AIcited by 25

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

also in 2 more
Abstract (arXiv)

Large language models (LLMs) exhibit remarkable problem-solving abilities, but struggle with complex tasks due to static internal knowledge. Retrieval-Augmented Generation (RAG) enhances access to external information, yet remains limited in multi-hop reasoning and strategic search due to rigid workflows. Recent advancements in agentic deep research empower LLMs to autonomously reason, search, and synthesize information. However, current approaches relying on outcome-based reinforcement learning (RL) face critical issues such as conflicting gradients and reward sparsity, limiting performance gains and training efficiency. To address these, we first propose Atomic Thought, a novel LLM thinking paradigm that decomposes reasoning into fine-grained functional units. These units are supervised by Reasoning Reward Models (RRMs), which provide Atomic Thought Rewards (ATR) for fine-grained guidance. Building on this, we propose Atom-Searcher, a novel RL framework for agentic deep research that integrates Atomic Thought and ATR. Atom-Searcher uses a curriculum-inspired reward schedule, prioritizing process-level ATR early and transitioning to outcome rewards, accelerating convergence on effective reasoning paths. Experiments on seven benchmarks show consistent improvements over the state-of-the-art. Key advantages include: (1) Atom-Searcher scales computation at test-time. (2) Atomic Thought provides supervision anchors for RRMs, bridging deep research tasks and RRMs. (3) Atom-Searcher exhibits more interpretable, human-like reasoning patterns.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 43 of its 59 references that are in the map. First posted on arXiv 18 Aug 2025.

Builds on

43 of 59 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capability30 Apr
  2. 2025Open Deep Search: Democratizing Search with Open-source Reasoning Agents26 Mar
  3. 2025ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning25 Mar
  4. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning12 Mar
  5. 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning7 Mar
  6. 2025AirRAG: Autonomous Strategic Planning and Reasoning Steer Retrieval Augmented Generation17 Jan
  7. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Models9 Jan
  8. 2023Retrieval-Augmented Generation for Large Language Models: A Survey18 Dec

Other references · 35

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2024Qwen2.5 Technical Reportcited by 4,942
  5. 2024GPT-4o System Cardcited by 4,588
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  7. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  8. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  9. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  10. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  11. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  12. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
23 more
  1. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  2. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  3. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  4. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
  5. 2025DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environmentscited by 286
  6. 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 256
  7. 2025Agentic Retrieval-Augmented Generation: A Survey on Agentic RAGcited by 249
  8. 2025Inference-Time Scaling for Generalist Reward Modelingcited by 239
  9. 2025ZeroSearch: Incentivize the Search Capability of LLMs without Searchingcited by 208
  10. 2024When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Methodcited by 204
  11. 2025WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learningcited by 165
  12. 2024Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAGcited by 123
  13. 2024Searching for Best Practices in Retrieval-Augmented Generationcited by 83
  14. 2024SuRe: Summarizing Retrievals using Answer Candidates for Open-domain QA of LLMscited by 67
  15. 2024Inference Scaling for Long-Context Retrieval Augmented Generationcited by 65
  16. 2024Auto-RAG: Autonomous Retrieval-Augmented Generation for Large Language Modelscited by 43
  17. 2025A Survey on Knowledge-Oriented Retrieval-Augmented Generationcited by 36
  18. 2024Measuring and Enhancing Trustworthiness of LLMs in RAG through Grounded Attributions and Learning to Refusecited by 34
  19. 2024OmniEval: An Omnidirectional and Automatic RAG Evaluation Benchmark in Financial Domaincited by 29
  20. 2024OpenResearcher: Unleashing AI for Accelerated Scientific Researchcited by 24
  21. 2024System for systematic literature review using multiple AI agents: Concept and an empirical evaluationcited by 13
  22. 2024CARROT: A Learned Cost-Constrained Retrieval Optimization System for RAGcited by 6
  23. 2025Careful Queries, Credible Results: Teaching RAG Models Advanced Web Search Tools with Reinforcement Learningcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 13; 2026: 12.

25 papers.

Sort:

By area:

  1. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
  2. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  3. 2025Information Gain-based Policy Optimization: A Simple and Effective Approach for Multi-Turn Search Agentscited by 48
  4. 2025Search Self-play: Pushing the Frontier of Agent Capability without Supervisioncited by 34
  5. 2025A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applicationscited by 22
  6. 2025Repurposing Synthetic Data for Fine-grained Search Agent Supervisioncited by 11
  7. 2025InfoFlow: Reinforcing Search Agent Via Reward Density Optimizationcited by 11
  8. 2025EviNote-RAG: Enhancing RAG Models via Answer-Supportive Evidence Notescited by 8
  9. 2026Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrievalcited by 8
  10. 2026Meta-Reinforcement Learning with Self-Reflection for Agentic Searchcited by 8
  11. 2025PRInTS: Reward Modeling for Long-Horizon Information Seekingcited by 5
  12. 2026SRR-Judge: Step-Level Rating and Refinement for Enhancing Search-Integrated Reasoning in Search Agentscited by 5
13 more
  1. 2025MR-Align: Meta-Reasoning Informed Factuality Alignment for Large Reasoning Modelscited by 4
  2. 2025Gradient Coupling: The Hidden Barrier to Generalization in Agentic Reinforcement Learningcited by 3
  3. 2025Code-driven Number Sequence Calculation: Enhancing the inductive Reasoning Abilities of Large Language Modelscited by 2
  4. 2026Adversarial Yet Cooperative: Multi-Perspective Reasoning in Retrieved-Augmented Language Modelscited by 2
  5. 2026Guided by Trajectories: Repairing and Rewarding Tool-Use Trajectories for Tool-Integrated Reasoningcited by 2
  6. 2025Agentic-KGR: Co-evolutionary Knowledge Graph Construction through Multi-Agent Reinforcement Learningcited by 1
  7. 2026PaperGuide: Making Small Language-Model Paper-Reading Agents More Efficientcited by 0
  8. 2026ProCeedRL: Process Critic with Exploratory Demonstration Reinforcement Learning for LLM Agentic Reasoningcited by 0
  9. 2026Contextual Information Policy Optimization for Search Agentscited by 0
  10. 2026Multi-Branch Policy Optimization for Multimodal Large Language Modelscited by 0
  11. 2026Making Every Tool Call Count: Necessary Tool-Evidence Path Rewards for Agentic Vision-Language Modelscited by 0
  12. 2026F2DR: A Fine-Grained Full-Pipeline Reward Framework for DeepSearch Workflowscited by 0
  13. 2026Search-Aware Reinforcement Learning for Multi-Component Query Understanding in Roblox Game Searchcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Beyond Correctness: Rewarding Faithful Reasoning in Retrieval-Augmented Generation
  2. 2026AgenticRag-R1: Agentic Reinforcement Learning with Stack Memory for Multi-Step Reasoning, Retrieval and Memorizing
  3. 2025MARAG-R1: Beyond Single Retriever via Reinforcement-Learned Multi-Tool Agentic Retrieval
  4. 2026Search More, Think Less: Rethinking Long-Horizon Agentic Search for Efficiency and Generalization
  5. 2026GRASP: GRanularity-Aware Search Policy for Agentic RAG
  6. 2026Debate as Reward: A Multi-Agent Reward System for Scientific Ideation via RL Post-Training
  7. 2025LeTS: Learning to Think-and-Search via Process-and-Outcome Reward Hybridization
  8. 2025HiPRAG: Hierarchical Process Rewards for Efficient Agentic Retrieval Augmented Generation