KARL: Knowledge Agents via Reinforcement Learning
On the map: LLM agents and tool use → Web, search, and research agents
Abstract (arXiv)
We present a system for training enterprise search agents via reinforcement learning that achieves state-of-the-art performance across a diverse suite of hard-to-verify agentic search tasks. Our work makes four core contributions. First, we introduce KARLBench, a multi-capability evaluation suite spanning six distinct search regimes, including constraint-driven entity search, cross-document report synthesis, tabular numerical reasoning, exhaustive entity retrieval, procedural reasoning over technical documentation, and fact aggregation over internal enterprise notes. Second, we show that models trained across heterogeneous search behaviors generalize substantially better than those optimized for any single benchmark. Third, we develop an agentic synthesis pipeline that employs long-horizon reasoning and tool use to generate diverse, grounded, and high-quality training data, with iterative bootstrapping from increasingly capable models. Fourth, we propose a new post-training paradigm based on iterative large-batch off-policy RL that is sample efficient, robust to train-inference engine discrepancies, and naturally extends to multi-task training with out-of-distribution generalization. Compared to Claude 4.6 and GPT 5.2, KARL is Pareto-optimal on KARLBench across cost-quality and latency-quality trade-offs, including tasks that were out-of-distribution during training. With sufficient test-time compute, it surpasses the strongest closed models. These results show that tailored synthetic data in combination with multi-task reinforcement learning enables cost-efficient and high-performing knowledge agents for grounded reasoning.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. First posted on arXiv 5 Mar 2026.
Builds on
0 of 0 references.
No references here yet.
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026OBLIQ-Bench: Exposing Overlooked Bottlenecks in Modern Retrievers with Latent and Implicit Queriescited by 5
- 2026AutoIndex: Learning Representation Programs for Retrievalcited by 0
- 2026PhantomEnvironments: Training LLM Agents in Fictional Worldscited by 0
- 2026ExperienceIndex: Artifact-Grounded Memorycited by 0
- 2026ExperienceIndex: Artifact-Grounded Memory7 Oct 2026
- 2026PhantomEnvironments: Training LLM Agents in Fictional Worlds30 Sep 2026
- 2026AutoIndex: Learning Representation Programs for Retrieval21 Jul 2026
- 2026OBLIQ-Bench: Exposing Overlooked Bottlenecks in Modern Retrievers with Latent and Implicit Queries7 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026CoSearch: Joint Training of Reasoning and Document Ranking via Reinforcement Learning for Agentic Search
- 2026SearchArt: Training Long-Horizon Search Agent with Scalable Synthetic and Verified Task
- 2025An Empirical Study on Reinforcement Learning for Reasoning-Search Interleaved LLM Agents
- 2025ReSeek: A Self-Correcting Framework for Search Agents with Instructive Rewards
- 2026PROGRESS: Coverage-guided RL to Train Search-augmented LLM Agent
- 2026Beyond Monolithic Architectures: A Multi-Agent Search and Knowledge Optimization Framework for Agentic Search
- 2025DSPO: Stable and Efficient Policy Optimization for Agentic Search and Reasoning
- 2025Beyond Ten Turns: Unlocking Long-Horizon Agentic Search with Large-Scale Asynchronous RL