mlsanityData to 14 Oct 2026

LiteResearcher: A Scalable Agentic RL Training Framework for Deep Research Agent

arXiv 2604.17931 ↗First posted 20 Apr 2026cs.AIcited by 10

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Reinforcement Learning (RL) has emerged as a powerful training paradigm for LLM-based agents. However, scaling agentic RL for deep research remains constrained by two coupled challenges: hand-crafted synthetic data fails to elicit genuine real-world search capabilities, and real-world search dependency during RL training introduces instability and prohibitive cost, which limits the scalability of Agentic RL. LiteResearcher is a training framework that makes Agentic RL scalable: by constructing a lite virtual world that mirrors real-world search dynamics, we enable a continuously improving training recipe that empowers a tiny search agent to outperform large-scale open-source and commercial models (e.g., Tongyi DeepResearch and Claude-4.5 Sonnet). Specifically, on common benchmarks such as GAIA and Xbench, our LiteResearcher-4B achieves open-source state-of-the-art results of 71.3% and 78.0% respectively, demonstrating that scalable RL training is a key enabler for Deep Research Agents.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 35 of its 44 references that are in the map. First posted on arXiv 20 Apr 2026.

Builds on

35 of 44 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling14 Nov
  2. 2025WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic Data and Scalable Reinforcement Learning16 Sep
  3. 2025WebExplorer: Explore and Evolve for Training Long-Horizon Web Agents8 Sep
  4. 2025Beyond Ten Turns: Unlocking Long-Horizon Agentic Search with Large-Scale Asynchronous RL11 Aug
  5. 2025WebDancer: Towards Autonomous Information Seeking Agency28 May
  6. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Models9 Jan

Other references · 29

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  6. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  7. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  8. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  9. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  10. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
  11. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
  12. 2025Process Reinforcement through Implicit Rewardscited by 504
17 more
  1. 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 467
  2. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 461
  3. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 335
  4. 2025DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environmentscited by 286
  5. 2025SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolutioncited by 263
  6. 2025ZeroSearch: Incentivize the Search Capability of LLMs without Searchingcited by 208
  7. 2025ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Modelscited by 194
  8. 2025WebWalker: Benchmarking LLMs in Web Traversalcited by 175
  9. 2024Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generationcited by 162
  10. 2025R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agentscited by 158
  11. 2025xbench: Tracking Agents Productivity Scaling with Profession-Aligned Real-World Evaluationscited by 120
  12. 2025Chain-of-Agents: End-to-End Agent Foundation Models via Multi-Agent Distillation and Agentic RLcited by 80
  13. 2025SealQA: Raising the Bar for Reasoning in Search-Augmented Language Modelscited by 75
  14. 2025MegaScience: Pushing the Frontiers of Post-Training Datasets for Science Reasoningcited by 59
  15. 2025SFR-DeepResearch: Towards Effective Reinforcement Learning for Autonomously Reasoning Single Agentscited by 31
  16. 2025Beyond Turn Limits: Training Deep Search Agents with Dynamic Context Windowcited by 19
  17. 2026AgentCPM-Explore: Realizing Long-Horizon Deep Exploration for Edge-Scale Agentscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 10.

10 papers.

  1. 2026Qwen-AgentWorld: Language World Models for General Agentscited by 28
  2. 2026SearchEyes: Towards Frontier Multimodal Deep Search Intelligence via Search World Simulationcited by 5
  3. 2026ResearchMath-14K: Scaling Research-Level Mathematics via Agentscited by 2
  4. 2026DuMate-DeepResearch: An Auditable Multi-Agent System with Recursive Search and Rubric-Grounded Reasoningcited by 2
  5. 2026MetaResearcher: Scaling Deep Research via Self-Reflective Reinforcement Learning in Adversarial Virtual Environmentscited by 1
  6. 2026AgenticRL: Agentic Reinforcement Learning with Self-Refinement for Complex UAV Navigationcited by 0
  7. 2026Training-Free Inference-Time Self-Reflection and Cost-Bounded Early Stopping for Large Language Modelscited by 0
  8. 2026Coding Agent Memory Post-training: Unlocking the Memory Potential of Pre-trained File Operations for Long-Horizon Tasks via Reinforcement Learningcited by 0
  9. 2026RIVO: Lightweight Adaptive Rerouting for Deep Research Agentscited by 0
  10. 2026Contextual Credit Policy Optimizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Long-MDR: Long-Context Reinforcement Learning for Multimodal Deep-Research Agents
  2. 2025Rethinking the Design of Reinforcement Learning-Based Deep Research Agents
  3. 2025Search Self-play: Pushing the Frontier of Agent Capability without Supervision
  4. 2026How to Train Your Deep Research Agent? Prompt, Reward, and Policy Optimization in Search-R1
  5. 2026Search More, Think Less: Rethinking Long-Horizon Agentic Search for Efficiency and Generalization
  6. 2026O-Researcher: An Open Ended Deep Research Model via Multi-Agent Distillation and Agentic RL
  7. 2025AgentRL: Scaling Agentic Reinforcement Learning with a Multi-Turn, Multi-Task Framework
  8. 2026PROGRESS: Coverage-guided RL to Train Search-augmented LLM Agent