mlsanityData to 7 Oct 2026

Cross-Domain Hybrid OPD for Generalizable Search Agents

arXiv 2608.02101 ↗First posted 3 Aug 2026cs.CLcited by 0

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Recent advances in Reinforcement Learning (RL) have substantially improved the capabilities of autonomous search agents, enabling sophisticated planning, and iterative retrieval over dynamic information sources. However, optimizing language models for specialized search behaviors often incurs an alignment tax, where gains in search performance come at the expense of general-purpose capabilities, limiting their effectiveness as universal assistants. In this technical report, we present the training framework behind the Yuanbao search agent, designed to achieve search specialization without sacrificing general intelligence. Built upon the Hunyuan3 architecture, our framework combines agentic reinforcement learning for autonomous search with a cross-domain expert On-Policy Distillation (OPD) pipeline. Experts specializing in complementary general-purpose domains are distilled into the search-specialized student, restoring and further enhancing its broad capabilities. Rather than treating specialization and general capability as competing objectives, our hybrid training strategy jointly optimizes both, effectively mitigating the alignment tax. Extensive experiments demonstrate that the resulting model achieves competitive search performance while consistently improving its general-purpose capabilities, providing a favorable balance between specialized execution and broad generalization in real-world search scenarios.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 33 of its 38 references that are in the map. First posted on arXiv 3 Aug 2026.

Builds on

33 of 38 references.

References · 33

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  7. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  8. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  9. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  10. 2021Measuring Coding Challenge Competence With APPScited by 879
  11. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 877
  12. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
21 more
  1. 2017Crowdsourcing Multiple Choice Science Questionscited by 742
  2. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  3. 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 502
  4. 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 322
  5. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 320
  6. 2025DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environmentscited by 268
  7. 2023RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentationcited by 253
  8. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 246
  9. 2025BIG-Bench Extra Hardcited by 123
  10. 2025Agentic Reasoning: A Streamlined Framework for Enhancing LLM Reasoning with Agentic Toolscited by 106
  11. 2023Self-Knowledge Guided Retrieval Augmentation for Large Language Modelscited by 69
  12. 2025Towards Robust Mathematical Reasoningcited by 67
  13. 2024MindSearch: Mimicking Human Minds Elicits Deep AI Searchercited by 65
  14. 2025SynLogic: Synthesizing Verifiable Reasoning Data at Scale for Learning Logical Reasoning and Beyondcited by 47
  15. 2026FrontierScience: Evaluating AI's Ability to Perform Expert-Level Scientific Taskscited by 42
  16. 2024Retrieve, Summarize, Plan: Advancing Multi-hop Question Answering with an Iterative Approachcited by 40
  17. 2025ScaleMCP: Dynamic and Auto-Synchronizing Model Context Protocol Tools for LLM Agentscited by 36
  18. 2025AMO-Bench: Large Language Models Still Struggle in High School Math Competitionscited by 36
  19. 2025PRBench: Large-Scale Expert Rubrics for Evaluating High-Stakes Professional Reasoningcited by 29
  20. 2025AutoCodeBench: Large Language Models are Automatic Code Benchmark Generatorscited by 24
  21. 2025SUPERChem: A Multimodal Reasoning Benchmark in Chemistrycited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applications
  2. 2026RAPO: Expanding Exploration for LLM Agents via Retrieval-Augmented Policy Optimization
  3. 2026PROGRESS: Coverage-guided RL to Train Search-augmented LLM Agent
  4. 2025An Empirical Study on Reinforcement Learning for Reasoning-Search Interleaved LLM Agents
  5. 2026KARL: Knowledge Agents via Reinforcement Learning
  6. 2026Paying Less Generalization Tax: A Cross-Domain Generalization Study of RL Training for LLM Agents
  7. 2025AI-SearchPlanner: Modular Agentic Search via Pareto-Optimal Multi-Objective Reinforcement Learning
  8. 2026From Proprietary to Open-Source: Bridging the Distribution Gap via Multi-Agent Protocol Distillation in Agentic Search