mlsanityData to 14 Oct 2026

Reinforcement Learning for Long-Horizon Multi-Turn Search Agents

arXiv 2510.24126 ↗First posted 28 Oct 2025cs.CLcited by 2

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

also in 1 more
Abstract (arXiv)

Large Language Model (LLM) agents can leverage multiple turns and tools to solve complex tasks, with prompt-based approaches achieving strong performance. This work demonstrates that Reinforcement Learning (RL) can push capabilities significantly further by learning from experience. Through experiments on a legal document search benchmark, we show that our RL-trained 14 Billion parameter model outperforms frontier class models (85% vs 78% accuracy). In addition, we explore turn-restricted regimes, during training and at test-time, that show these agents achieve better results if allowed to operate over longer multi-turn horizons.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 14 of its 24 references that are in the map. First posted on arXiv 28 Oct 2025.

Builds on

14 of 24 references.

References · 14

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  7. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  8. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 981
  9. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 601
  10. 2023Active Retrieval Augmented Generationcited by 516
  11. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMscited by 289
  12. 2024A Review of Prominent Paradigms for LLM-Based Agents: Tool Use (Including RAG), Planning, and Feedback Learningcited by 74
2 more
  1. 2024Offline Reinforcement Learning for LLM Multi-Step Reasoningcited by 54
  2. 2025A Survey on LLM Test-Time Compute via Search: Tasks, LLM Profiling, Search Algorithms, and Relevant Frameworkscited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026ARCO: Adaptive Rubrics with Co-Evolution for Multi-Step LLM-Based Agentscited by 4
  2. 2026CHART: A Harness-Rotation Curriculum for Harness-Robust Search Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Information Gain-based Policy Optimization: A Simple and Effective Approach for Multi-Turn Search Agents
  2. 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL
  3. 2025An Empirical Study on Reinforcement Learning for Reasoning-Search Interleaved LLM Agents
  4. 2025Beyond Ten Turns: Unlocking Long-Horizon Agentic Search with Large-Scale Asynchronous RL
  5. 2026TSR: Trajectory-Search Rollouts for Multi-Turn RL of LLM Agents
  6. 2025Scaling LLM Multi-turn RL with End-to-end Summarization-based Context Management
  7. 2025Reinforcing Multi-Turn Reasoning in LLM Agents via Fine-Grained Reward Structure and Credit Assignment
  8. 2026From Player to Master: Enhancing Test-Time Learning of LLM Agents via Reinforcement Learning over Memory