Cross-Domain Hybrid OPD for Generalizable Search Agents
On the map: LLM agents and tool use → Web, search, and research agents
Abstract (arXiv)
Recent advances in Reinforcement Learning (RL) have substantially improved the capabilities of autonomous search agents, enabling sophisticated planning, and iterative retrieval over dynamic information sources. However, optimizing language models for specialized search behaviors often incurs an alignment tax, where gains in search performance come at the expense of general-purpose capabilities, limiting their effectiveness as universal assistants. In this technical report, we present the training framework behind the Yuanbao search agent, designed to achieve search specialization without sacrificing general intelligence. Built upon the Hunyuan3 architecture, our framework combines agentic reinforcement learning for autonomous search with a cross-domain expert On-Policy Distillation (OPD) pipeline. Experts specializing in complementary general-purpose domains are distilled into the search-specialized student, restoring and further enhancing its broad capabilities. Rather than treating specialization and general capability as competing objectives, our hybrid training strategy jointly optimizes both, effectively mitigating the alignment tax. Extensive experiments demonstrate that the resulting model achieves competitive search performance while consistently improving its general-purpose capabilities, providing a favorable balance between specialized execution and broad generalization in real-world search scenarios.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 33 of its 38 references that are in the map. First posted on arXiv 3 Aug 2026.
Builds on
33 of 38 references.
References · 33
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2025Qwen3 Technical Reportcited by 8,195
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
- 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
- 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
- 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
- 2021Measuring Coding Challenge Competence With APPScited by 879
- 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 877
- 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
21 more
- 2017Crowdsourcing Multiple Choice Science Questionscited by 742
- 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
- 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 502
- 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 322
- 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 320
- 2025DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environmentscited by 268
- 2023RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentationcited by 253
- 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 246
- 2025BIG-Bench Extra Hardcited by 123
- 2025Agentic Reasoning: A Streamlined Framework for Enhancing LLM Reasoning with Agentic Toolscited by 106
- 2023Self-Knowledge Guided Retrieval Augmentation for Large Language Modelscited by 69
- 2025Towards Robust Mathematical Reasoningcited by 67
- 2024MindSearch: Mimicking Human Minds Elicits Deep AI Searchercited by 65
- 2025SynLogic: Synthesizing Verifiable Reasoning Data at Scale for Learning Logical Reasoning and Beyondcited by 47
- 2026FrontierScience: Evaluating AI's Ability to Perform Expert-Level Scientific Taskscited by 42
- 2024Retrieve, Summarize, Plan: Advancing Multi-hop Question Answering with an Iterative Approachcited by 40
- 2025ScaleMCP: Dynamic and Auto-Synchronizing Model Context Protocol Tools for LLM Agentscited by 36
- 2025AMO-Bench: Large Language Models Still Struggle in High School Math Competitionscited by 36
- 2025PRBench: Large-Scale Expert Rubrics for Evaluating High-Stakes Professional Reasoningcited by 29
- 2025AutoCodeBench: Large Language Models are Automatic Code Benchmark Generatorscited by 24
- 2025SUPERChem: A Multimodal Reasoning Benchmark in Chemistrycited by 7
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applications
- 2026RAPO: Expanding Exploration for LLM Agents via Retrieval-Augmented Policy Optimization
- 2026PROGRESS: Coverage-guided RL to Train Search-augmented LLM Agent
- 2025An Empirical Study on Reinforcement Learning for Reasoning-Search Interleaved LLM Agents
- 2026KARL: Knowledge Agents via Reinforcement Learning
- 2026Paying Less Generalization Tax: A Cross-Domain Generalization Study of RL Training for LLM Agents
- 2025AI-SearchPlanner: Modular Agentic Search via Pareto-Optimal Multi-Objective Reinforcement Learning
- 2026From Proprietary to Open-Source: Bridging the Distribution Gap via Multi-Agent Protocol Distillation in Agentic Search