# MaskSearch: A Universal Pre-Training Framework to Enhance Agentic Search Capability

arXiv: https://arxiv.org/abs/2505.20285 · HTML: https://mlsanity.com/paper/2505.20285/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Agentic RAG, agents and memory, one of 15 directions in Retrieval-augmented generation.
- Builds on 4 of its 5 references that are in the map.
- First posted on arXiv 26 May 2025.

## Placed in

- Retrieval-augmented generation / [Agentic RAG, agents and memory](https://mlsanity.com/rag/agentic-rag-agents-and-memory/)
- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 4 of 4)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Multiagent Finetuning: Self Improvement with Diverse Reasoning Chains](https://mlsanity.com/paper/2501.05707/) (2025)
- [START: Self-taught Reasoner with Tools](https://mlsanity.com/paper/2503.04625/) (2025)
- [Question-Analysis Prompting Improves LLM Performance in Reasoning Tasks](https://mlsanity.com/paper/2407.03624/) (2024)

## Built on by (most cited first, 22 of 22)

- [WebShaper: Agentically Data Synthesizing via Information-Seeking Formalization](https://mlsanity.com/paper/2507.15061/) (2025)
- [Deep Research: A Survey of Autonomous Research Agents](https://mlsanity.com/paper/2508.12752/) (2025)
- [A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challenges](https://mlsanity.com/paper/2508.05668/) (2025)
- [WebWeaver: Structuring Web-Scale Evidence with Dynamic Outlines for Open-Ended Deep Research](https://mlsanity.com/paper/2509.13312/) (2025)
- [Scaling Agents via Continual Pre-training](https://mlsanity.com/paper/2509.13310/) (2025)
- [ParallelSearch: Train your LLMs to Decompose Query and Search Sub-queries in Parallel with Reinforcement Learning](https://mlsanity.com/paper/2508.09303/) (2025)
- [A Survey on Agentic Multimodal Large Language Models](https://mlsanity.com/paper/2510.10991/) (2025)
- [A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applications](https://mlsanity.com/paper/2510.16724/) (2025)
- [Reasoning and Tool-use Compete in Agentic RL:From Quantifying Interference to Disentangled Tuning](https://mlsanity.com/paper/2602.00994/) (2026)
- [RAG-R1: Incentivizing the Search and Reasoning Capabilities of LLMs through Multi-query Parallelism](https://mlsanity.com/paper/2507.02962/) (2025)
- [APTBench: Benchmarking Agentic Potential of Base LLMs During Pre-Training](https://mlsanity.com/paper/2510.24397/) (2025)
- [Beyond Pipelines: A Survey of the Paradigm Shift toward Model-Native Agentic AI](https://mlsanity.com/paper/2510.16720/) (2025)
- [RhinoInsight: Improving Deep Research through Control Mechanisms for Model Behavior and Context](https://mlsanity.com/paper/2511.18743/) (2025)
- [Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application](https://mlsanity.com/paper/2606.12191/) (2026)
- [Erase to Improve: Erasable Reinforcement Learning for Search-Augmented LLMs](https://mlsanity.com/paper/2510.00861/) (2025)
- [Beyond the limitation of a single query: Train your LLM for query expansion with Reinforcement Learning](https://mlsanity.com/paper/2510.10009/) (2025)
- [DocArena: Turning Raw Documents into Controllable Training Environments for Document Search Agents](https://mlsanity.com/paper/2606.26122/) (2026)
- [Yet Even Less Is Even Better For Agentic, Reasoning, and Coding LLMs](https://mlsanity.com/paper/2604.00824/) (2026)
- [SAAS: Self-Aware Reinforcement Learning for Over-Search Mitigation in Agentic Search](https://mlsanity.com/paper/2605.29796/) (2026)
- [SubSearch: Intermediate Rewards for Unsupervised Guided Reasoning in Complex Retrieval](https://mlsanity.com/paper/2604.07415/) (2026)
- [SPT: Skills as Pre-Training Data for Agentic Language Models](https://mlsanity.com/paper/2608.26563/) (2026)
- [F2DR: A Fine-Grained Full-Pipeline Reward Framework for DeepSearch Workflows](https://mlsanity.com/paper/2609.19827/) (2026)

## Abstract (arXiv)

Retrieval-Augmented Language Models (RALMs) represent a classic paradigm where models enhance generative capabilities using external knowledge retrieved via a specialized module. Recent advancements in Agent techniques enable Large Language Models (LLMs) to autonomously utilize tools for retrieval, planning, and reasoning. While existing training-based methods show promise, their agentic abilities are limited by inherent characteristics of the task-specific data used during training. To further enhance the universal search capability of agents, we propose a novel pre-training framework, MaskSearch. In the pre-training stage, we introduce the Retrieval Augmented Mask Prediction (RAMP) task, where the model learns to leverage search tools to fill masked spans on a large number of pre-training data, thus acquiring universal retrieval and reasoning capabilities for LLMs. After that, the model is trained on downstream tasks to achieve further improvement. We apply both Supervised Fine-tuning (SFT) and Reinforcement Learning (RL) for training. For SFT, we combine agent-based and distillation-based methods to generate training data, starting with a multi-agent system consisting of a planner, rewriter, observer, and followed by a self-evolving teacher model. While for RL, we employ DAPO as the training framework and adopt a hybrid reward system consisting of answer rewards and format rewards. Additionally, we introduce a curriculum learning approach that allows the model to learn progressively from easier to more challenging instances based on the number of masked spans. We evaluate the effectiveness of our framework in the scenario of open-domain multi-hop question answering. Through extensive experiments, we demonstrate that MaskSearch significantly enhances the performance of LLM-based search agents on both in-domain and out-of-domain downstream tasks.