mlsanityData to 7 Oct 2026

Reinforcement learning for LLM agents

850 papers16 this weekMedium confidence in this grouping: mostly one topic, with blurry edgesMedium confidence in this grouping: mostly one topic, with blurry edges · About

Keywords: reinforcement, reinforcement learning, training, agents, policy

Key facts. Reinforcement learning for LLM agents: 850 papers on arXiv since 2022 (data through 7 Oct 2026). 159 of its 850 mapped papers (19%) cite “Reflexion: Language Agents with Verbal Reinforcement Learning” (2023, arXiv 2303.11366). 151 of its 850 mapped papers (18%) cite “Group-in-Group Policy Optimization for LLM Agent Training” (2025, arXiv 2505.10978). 112 new papers 17 Sep–7 Oct.

What’s new

17 Sep – 7 Oct 2026

112 new papers, 17 Sep – 7 Oct; 16 this week.

  1. This week · 16
  2. StoreBench: A Live-Commerce Environment for Evaluating and Training Autonomous Operator Agents also fits: General LLM agent frameworks and benchmarks
  3. Beyond Outcome Rewards: Constructing and Assigning Retrieval Credit for Search Agents
  4. Training Advisors for LLM Agents from Task Outcomes also fits: General LLM agent frameworks and benchmarks
  5. From Pareto to Preference: Personalized Test-Time Scaling via Amortized Agentic Policy Discovery also fits: General LLM agent frameworks and benchmarks
  6. MIMESIS: Learning User Simulators as Training Environments for Interactive Agents also fits: General LLM agent frameworks and benchmarks
  7. FreeEvolve: Learning to Evolve Beyond Fixed Loops also fits: Agent skill learning and evolution
  8. From Uncertainty to Action: Learning to Steer LLM Agents also fits: Agent skill learning and evolution
  9. TAP: Efficient Long-Horizon Agent Pruning via Trajectory-Anchored Recovery
  10. Verify Less, Evolve More: Training Idea-Level Critics for Verification-Efficient ML Evolving Agents also fits: General LLM agent frameworks and benchmarks
  11. GraphOPD: Graph-Augmented On-Policy Distillation for LLM Agents
All 112
  1. NeMo-DCR: Bit-Exact Delta-Compressed Refit for Scalable Agentic RL at Trillion-Parameter Scale
  2. VETTA: Coordinating Turn- and Token-Level Credit Assignment for Multi-Turn LLM Agents
  3. Selective Critique for Cost-Aware LLM Agents in Long-Horizon Decision Making also fits: General LLM agent frameworks and benchmarks
  4. Structuring MoE Expert Selection for Agentic Reinforcement Learning
  5. Do Small Language Models Learn to Negotiate? A Controlled Scaling Study of RL-Trained Sellers also fits: Financial trading and forecasting agents
  6. ThunderSyncRL: Lossless Acceleration of Agentic Reinforcement Learning
  7. Earlier · 96
  8. Hierarchical Reinforcement Learning with Stable Temporal Abstraction for Language Model Agents
  9. Long-MDR: Long-Context Reinforcement Learning for Multimodal Deep-Research Agents
  10. Rewrite What Matters: Adaptive Multilingual Query Rewriting for Reasoning via Agentic Reinforcement Learning also fits: General LLM agent frameworks and benchmarks
  11. ForkPilot: Self-Evolving Policy for Retrospective Search in Long-Horizon Agents
  12. Can LLM Agents Automate Reinforcement Learning for Text-to-Speech? also fits: General LLM agent frameworks and benchmarks
  13. Asynchronous Is Nearly Free for Evolution Strategies on Long-Horizon Agentic Tasks
  14. Follow the Winners: Conservative Policy Improvement with the Cross-Entropy Method for Critic-Free RFT
  15. AdaStep: Adaptive Step Credit Weighting for Agentic Reinforcement Learning
  16. LEAP: Learning Efficient Action Proposals For LLM Agents also fits: General LLM agent frameworks and benchmarks
  17. OpenGameEval: Benchmarking Agentic Programming and Exploration in a Stateful Game Engine also fits: Agent skill learning and evolution
  18. Test-time Multi-agent Coordination by Decomposed Value Gradient Flow
  19. Finding the Move Is Not Winning the Game: XiangqiBench for Closed-Loop Evaluation of LLM Agents also fits: General LLM agent frameworks and benchmarks
  20. Sharpening Tax in Post-Training
  21. Dependency-Aware Reward Shaping for Agentic Reinforcement Learning
  22. My FAULT: Self-Diagnosis as Credit Assignment in Self-Evolving Agentic Reinforcement Learning
  23. RISED: RubrIcs for agentic multi-environment Selection and sElf-Distillation
  24. ActiveSaddler: Automated Curriculum Learning for Agent Harness Optimization also fits: Agent skill learning and evolution
  25. Cross-Benchmark Transfer from RL on Agentic Coding Tasks also fits: Agent skill learning and evolution
  26. ComputerSD: Online Self-Distillation from Real-Time Feedback for Computer-Use Agents
  27. EVOKE: Eliciting World Knowledge in Agents for Transferable Decision-Making also fits: General LLM agent frameworks and benchmarks
  28. SHARPO: Segment-Level Credit Assignment for Agentic Reinforcement Learning
  29. No One Architecture Fits All: A Cross-Environment Evaluation of Hierarchical Red Team Agents also fits: Agent threats, guardrails and governance
  30. T2SPO: Trajectory-to-Step Policy Optimization for Agentic Reinforcement Learning
  31. When Harnesses Lose the Signal: Causal Evaluation of Recovery in LLM Agents also fits: General LLM agent frameworks and benchmarks
  32. PhantomEnvironments: Training LLM Agents in Fictional Worlds
  33. OverForge: Reasoning Through Strategies and Tactics Helps Cooperative Lifelong Adaptation also fits: General LLM agent frameworks and benchmarks
  34. From Imitation to Reward Discovery: On-Policy Warmup for Agentic RL
  35. Beyond Prediction: Steering VLM Agents with Retrospective World Modeling
  36. RSIGame: Autonomous Agentic Game Development with Recursive Self-improvement also fits: Software engineering coding agents
  37. STRATA: Self-Learning Through Role-Aligned Tiered Agents for Real-Time Strategy Games
  38. Explicit Trajectory Diversity for RL-Based Post-Training of LLM Agents
  39. Learning from Viable Failure Prefixes: Milestone Viability Potential Policy Optimization for Long-Horizon LLM Agents
  40. UserProxyBench: Evaluating LLM User Simulators for Agent Benchmarks and Training also fits: General LLM agent frameworks and benchmarks
  41. Guide, Then Let Go: Gap-Adaptive Teacher Scheduling for Sparse-Reward Agentic RL
  42. VACE: Validation-Gated Alternating Co-Evolution of Agent Models and Harnesses
  43. PR-OPD: Privileged Representation On-policy Self-Distillation for Agentic Reinforcement Learning
  44. BRIDGE: Bilevel Retrieval-Credit-Aware Agentic Reinforcement Learning
  45. KV-streams for Efficient Compaction in Agentic Reinforcement Learning
  46. RLE-Bench: A Qualifying Exam for Coding Agents as Robot Learning Engineers also fits: General LLM agent frameworks and benchmarks
  47. Reward-rate Policy Gradient for Efficient Machine Learning Engineering Agents
  48. Targeting Pivotal Decisions for Credit Assignment in Agentic Reinforcement Learning
  49. Shockingly Simple Self-retrospection Improves Agentic Models Without RL also fits: Agent skill learning and evolution
  50. Harness Learning Enables Generalizable Test-Time Adaptation also fits: Agent skill learning and evolution
  51. ARISE: Adapting to Evolving Capability Gaps in Agentic Reinforcement Learning also fits: Agent skill learning and evolution
  52. PIVOT: Pivot-Aware On Policy Self Distillation for Multi-Turn VLM Agents
  53. ASCT: Attentive Search over Counterfactual Trees for Credit Assignment in Agentic Reinforcement Learning
  54. PEARL: Adaptive Prefill-Decode Execution with Elasticity for Agentic Reinforcement Learning
  55. GraphHCA: Closed-Form Hindsight Credit Assignment for Long-Horizon LLM Agents
  56. Cross-Rollout Bellman Closure for Long-Horizon Agentic Reinforcement Learning
  57. TIDE: Teacher-Student Transition via Informative Distillation and Exploration for Agentic RL
  58. UniOPSD: Unifying Outcome and Hindsight Feedback for Agentic Reinforcement Learning
  59. SIPO: Selective-Inference Policy Optimization for Tree-Structured Agentic RL
  60. ARS: Agentic Reward System for Robot Learning
  61. USA: Update-aware SAM for Cross-domain On-Policy Disitllation of Language Agents also fits: General LLM agent frameworks and benchmarks
  62. Same Winners, Different Success Rates: Evaluating How LLM Agents Recover from Failures also fits: General LLM agent frameworks and benchmarks
  63. GlyphBench: A Playground for Language-Model Reinforcement Learning
  64. Learning Perturbation Robust Policies for LLM Agents with Stable Optimization
  65. HyperMCTS: Hypergraph-Augmented MCTS for Long-Horizon LLM Agents
  66. QwenGyre: An Elastic Reinforcement Learning Framework for Training xLong-Horizon Agents
  67. Trajectory Unlearning on LLM-based Agents
  68. ParaAgent: Reinforcing Parallel Acting in Open-World Tool Environments also fits: General LLM agent frameworks and benchmarks
  69. NLPG: Natural-Language Policy Gradients for Self-Evolving Language Agents
  70. Agentic Multi-Turn Reasoning: A Fairness Approach also fits: General LLM agent frameworks and benchmarks
  71. CodeSkill: Latent Skill Abstraction for Long-Horizon Code Agents also fits: Agent skill learning and evolution
  72. Downstream-Aware Context Selection for Online In-Context Reinforcement Learning
  73. Improving LLM Collaboration via Multi-Agent Preference Learning also fits: General LLM agent frameworks and benchmarks
  74. CUA-Sandbox: Efficient Environments for Computer-Use Agent Reinforcement Learning also fits: GUI and mobile agents
  75. Porimon: An LLM-Based Pokémon Battle Agent Enhanced by Long/Short-Term Knowledge Augmented Generation
  76. Multi-Agent System Search via Active Substructure-aware Policy Optimization also fits: General LLM agent frameworks and benchmarks
  77. Beyond Scripted Search: Sample-Efficient Reward Discovery via Agentic Black-box Optimization
  78. Train4Merge: A Controlled Single-Teacher Study of RL vs. SFT Teachers for OPD-Based Model Merging also fits: Educational agents and math problem solving
  79. Find Something You Can't Do: Agentic Real-World Reinforcement Learning for Self-Improving VLA Models also fits: Multimodal and embodied agents
  80. Before the Rollout Ends: Early Terminal Reward Prediction for Long-horizon Coding Agents
  81. Coding Agents for Generalized Task and Motion Planning Problems also fits: Multimodal and embodied agents
  82. RACaP: Agentic Reasoning, Acting, and Coding as Policies for Evolvable Robot Learning also fits: Agent skill learning and evolution
  83. From Self-Distillation to Self-Practice: Privileged Information for Multi-Turn Agents also fits: Agent skill learning and evolution
  84. SLCA-GRPO: Resolving Cross-Segment Credit Misattribution in Tool-Calling RL
  85. Back to the Definition: Estimating Step-Level Advantages via Trajectory Graphs for Agentic Reinforcement Learning
  86. Adversarial Closed-Loop Curriculum for Evolving Role-Playing Agents also fits: Agent skill learning and evolution
  87. Pistis Technical Report
  88. State-Grounded Conditioning: Wrapping User-Facing LLM Agents Where Direction Depends on Live State
  89. ProCredit: From Outcome Rewards to Progress Credit in Agentic Reinforcement Learning
  90. Reinforcement Learning with Decomposed Subtasks
  91. MAGIC: Mixed-Granularity Agent Graphs via Incremental Construction with Dense-Reward Reinforcement Learning
  92. Harness-Zero: Harness Distillation via Agent-as-Harness also fits: Agent skill learning and evolution
  93. MimicAgent: Quadruped Skills via Prompt-to-Trajectory Generation also fits: Agent skill learning and evolution
  94. Luck Is Not Skill: When Do Paired Rollouts Help Group-Relative RL of LLM Agents?
  95. Synthesizing Reactive Character Behaviors for Continuous Games via Programmatic Policy Search
  96. Towards Full Pipeline FP8 Reinforcement Learning for LLMs
  97. AutoGym: Blueprint-First Generation of Verifiable Agent Gyms also fits: General LLM agent frameworks and benchmarks
  98. ArenaFlow: From Trajectory Ranking to Hierarchical Credit Propagation for Open-Ended Agent RL
  99. RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement Learning
  100. UnifiedPlayers: Enhance Tool-Integrated Reasoning in Agentic Reinforcement Learning
  101. Dual-Axis Policy Optimization for LLM Agents: Bayesian Feedback Attribution and Trajectory Mass Normalization
  102. DeliveryGym: An RL Environment for Long-Horizon Embodied Agent Planning with Adaptive Curriculum
  103. Reach or Solve? Attributing Agentic RL Gains with Checkpoint Handoffs

Foundations

Most cited by papers here

  1. Reflexion: Language Agents with Verbal Reinforcement Learning2023cited by 159 here
  2. Group-in-Group Policy Optimization for LLM Agent Training2025cited by 151 here
  3. Voyager: An Open-Ended Embodied Agent with Large Language Models2023cited by 135 here
  4. RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning2025cited by 103 here
  5. ReTool: Reinforcement Learning for Strategic Tool Use in LLMs2025cited by 86 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 65 here
  2. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 52 here
  3. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 135 here
  4. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 67 here
  5. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 42 here
  6. 2025ToolRL: Reward is All Tool Learning Needscited by 84 here
  7. 2025Group-in-Group Policy Optimization for LLM Agent Trainingcited by 151 here
  8. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 52 here
16 more it builds on, not in this chain
  1. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 151 here
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 136 here
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 159 here
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 84 here
  5. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 75 here
  6. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 56 here
  7. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 43 here
  8. 2023A Survey on Large Language Model based Autonomous Agentscited by 66 here
  9. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 46 here
  10. 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 50 here
  11. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 47 here
  12. 2025ToRL: Scaling Tool-Integrated RLcited by 51 here
  13. 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMscited by 86 here
  14. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 103 here
  15. 2025Agentic Reinforced Policy Optimizationcited by 49 here
  16. 2025SimpleTIR: End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoningcited by 42 here

The 24 most cited of 1,643.

Browse

30 of 850, ranked 6–35

  1. 6ToolRL: Reward is All Tool Learning Needs202584
  2. 7MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge202265
  3. 8The Landscape of Agentic Reinforcement Learning for LLMs: A Survey202552
  4. 9ToRL: Scaling Tool-Integrated RL202551
  5. 10ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL202450
  6. 11Agentic Reinforced Policy Optimization202549
  7. 12SimpleTIR: End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoning202542
  8. 13Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory202335
  9. 14JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models202334
  10. 15Trial and Error: Exploration-Based Trajectory Optimization for LLM Agents202431
All 30
  1. 16Reinforcement Learning for Long-Horizon Interactive LLM Agents202529
  2. 17Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Game202329
  3. 18Agentic Reasoning and Tool Integration for LLMs via Reinforcement Learning202528
  4. 19SPA-RL: Reinforcing LLM Agents via Stepwise Progress Attribution202527
  5. 20Agent Lightning: Train ANY AI Agents with Reinforcement Learning202527
  6. 21VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Use202526
  7. 22Reinforcing Multi-Turn Reasoning in LLM Agents via Fine-Grained Reward Structure and Credit Assignment202524
  8. 23Large Language Models Play StarCraft II: Benchmarks and A Chain of Summarization Approach202324
  9. 24Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning202423
  10. 25Process Reward Models for LLM Agents: Practical Framework and Directions202522
  11. 26MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learning202522
  12. 27AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progress202522
  13. 28Agent RL Scaling Law: Agent RL with Spontaneous Code Execution for Mathematical Problem Solving202521
  14. 29SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learning202521
  15. 30Tree Search for LLM Agent Reinforcement Learning202521
  16. 31BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games202421
  17. 32Hierarchy-of-Groups Policy Optimization for Long-Horizon Agentic Tasks202620
  18. 33SWEET-RL: Training Multi-Turn LLM Agents on Collaborative Reasoning Tasks202520
  19. 34Agentic Reinforcement Learning with Implicit Step Rewards202520
  20. 35AvalonBench: Evaluating LLMs Playing the Game of Avalon202320