Data to 14 Oct 2026

Agent Lightning: Train ANY AI Agents with Reinforcement Learning

arXiv 2508.03680 ↗First posted 5 Aug 2025cs.AI cs.LGcited by 109

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

also in 1 more

Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

We present Agent Lightning, a flexible and extensible framework that enables Reinforcement Learning (RL)-based training of Large Language Models (LLMs) for any AI agent. Unlike existing methods that tightly couple RL training with agent or rely on sequence concatenation with masking, Agent Lightning achieves complete decoupling between agent execution and training, allowing seamless integration with existing agents developed via diverse ways (e.g., using frameworks like LangChain, OpenAI Agents SDK, AutoGen, and building from scratch) with almost ZERO code modifications. By formulating agent execution as Markov decision process, we define an unified data interface and propose a hierarchical RL algorithm, LightningRL, which contains a credit assignment module, allowing us to decompose trajectories generated by ANY agents into training transition. This enables RL to handle complex interaction logic, such as multi-agent scenarios and dynamic workflows. For the system design, we introduce a Training-Agent Disaggregation architecture, and brings agent observability frameworks into agent runtime, providing a standardized agent finetuning interface. Experiments across text-to-SQL, retrieval-augmented generation, and math tool-use tasks demonstrate stable, continuous improvements, showcasing the framework's potential for real-world agent training and deployment.

Key facts. Cited by 109 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 30 of its 52 references that are in the map. First posted on arXiv 5 Aug 2025.

Builds on

30 of 52 references.

Closest ancestors · 3

Same direction, newest first

Other references · 27

2022Training language models to follow instructions with human feedbackcited by 15,971
2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139

15 more

2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 567
2017Ray: A Distributed Framework for Emerging AI Applicationscited by 527
2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 383
2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 336
2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learningcited by 262
2023FireAct: Toward Language Agent Fine-tuningcited by 256
2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoningcited by 250
2024Large Language Model-Based Agents for Software Engineering: A Surveycited by 178
2025Reinforcement Learning Optimization for Large-Scale Learning: An Efficient and User-Friendly Scaling Librarycited by 101
2024Parrot: Efficient Serving of LLM-based Applications with Semantic Variablecited by 99
2023A Multi-Task Embedder For Retrieval Augmented LLMscited by 64
2025A Comprehensive Survey of Deep Research: Systems, Methodologies, and Applicationscited by 50
2025Trinity-RFT: A General-Purpose and Unified Framework for Reinforcement Fine-Tuning of Large Language Modelscited by 19
2023Calc-X and Calcformers: Empowering Arithmetical Chain-of-Thought through Interaction with Symbolic Systemscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 19; 2026: 90.

109 papers.

Sort:

By area:

2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
2026Hierarchy-of-Groups Policy Optimization for Long-Horizon Agentic Taskscited by 49
2025SkyRL-Agent: Efficient RL Training for Multi-turn LLM Agentcited by 48
2026SEED: Self-Evolving On-Policy Distillation for Agentic Reinforcement Learningcited by 32
2026Beyond Trajectory-Level Attribution: Graph-Based Credit Assignment for Agentic Reinforcement Learningcited by 31
2025Agent-R1: A Unified and Modular Framework for Agentic Reinforcement Learningcited by 29
2026From Reasoning to Agentic: Credit Assignment in Reinforcement Learning for Large Language Modelscited by 23
2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skillscited by 22
2026From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanismscited by 21
2026ProRL Agent: Rollout-as-a-Service for RL Training of Multi-Turn LLM Agentscited by 20
2026Exploratory Memory-Augmented LLM Agent via Hybrid On- and Off-Policy Optimizationcited by 19

38 more

2025On the Faithfulness of Visual Thinking: Measurement and Enhancementcited by 17
2025AgentMath: Empowering Mathematical Reasoning for Large Language Models via Tool-Augmented Agentcited by 17
2026Dr. MAS: Stable Reinforcement Learning for Multi-Agent LLM Systemscited by 16
2026MemMachine: A Ground-Truth-Preserving Memory System for Personalized AI Agentscited by 16
2025Reinforcement Learning Foundations for Deep Research Systems: A Surveycited by 14
2025GTM: Simulating the World of Tools for AI Agentscited by 14
2026PivotRL: High Accuracy Agentic Post-Training at Low Compute Costcited by 14
2026Group-Graph Policy Optimization for Long-Horizon Agentic Reinforcement Learningcited by 13
2026RubricEM: Meta-RL with Rubric-guided Policy Decomposition beyond Verifiable Rewardscited by 12
2026DeltaBox: Scaling Stateful AI Agents with Millisecond-Level Sandbox Checkpoint/Rollbackcited by 12
2025Confucius Code Agent: Scalable Agent Scaffolding for Real-World Codebasescited by 11
2025Memory-T1: Reinforcement Learning for Temporal Reasoning in Multi-session Agentscited by 11
2025ResT: Reshaping Token-Level Policy Gradients for Tool-Use Large Language Modelscited by 10
2025Beyond Pipelines: A Survey of the Paradigm Shift toward Model-Native Agentic AIcited by 8
2025Audited Skill-Graph Self-Improvement for Agentic LLMs via Verifiable Rewards, Experience Synthesis, and Continual Memorycited by 8
2026TSR: Trajectory-Search Rollouts for Multi-Turn RL of LLM Agentscited by 7
2026Agent Lightning v1.0: Towards Harnessed Agentic RLcited by 7
2026On Training Large Language Models for Long-Horizon Tasks: An Empirical Study of Horizon Lengthcited by 6
2026TRUSTMEM: Learning Trustworthy Memory Consolidation for LLM Agents with Long-Term Memorycited by 6
2025Tree Training: Accelerating Agentic LLMs Training via Shared Prefix Reusecited by 5
2025Youtu-Agent: Scaling Agent Productivity with Automated Generation and Hybrid Policy Optimizationcited by 5
2026PaperScout: An Autonomous Agent for Academic Paper Search with Process-Aware Sequence-Level Policy Optimizationcited by 5
2026Reinforcement Learning for LLM-based Multi-Agent Systems through Orchestration Tracescited by 5
2026SkillRise: Agentic Reinforcement Learning for Cross-Task Skill Evolutioncited by 5
2025SeamlessFlow: A Trainer Agent Isolation RL Framework Achieving Bubble-Free Pipelines via Tag Schedulingcited by 4
2025Benefits and Pitfalls of Reinforcement Learning for Language Model Planning: A Theoretical Perspectivecited by 4
2026StraTA: Incentivizing Agentic Reinforcement Learning with Strategic Trajectory Abstractioncited by 4
2026Intern-S2-Preview: Scientific Agentic Foundation Modelcited by 4
2026LEGO-RL: Harness-Native Reinforcement Learning for Coding Agentscited by 4
2026OpenTinker: Separating Concerns in Agentic Reinforcement Learningcited by 3
2026Not All Turns Matter: Credit Assignment for Multi-Turn Jailbreakingcited by 3
2026PACT: Privileged Trace Co-Training for Multi-Turn Tool-Use Agentscited by 3
2026AgentHPOBench: A Benchmark For Evaluating LLM Agents as Sequential Hyperparameter Optimizerscited by 3
2026Verifiable Memory: Learning Unified Memory Management with Local and Global Verifiers for Large Language Model Agentscited by 3
2026Agentic Reinforcement Learning with Self-Distilled Reward Shapingcited by 3
2026Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligencecited by 3
2025ToolBrain: A Flexible Reinforcement Learning Framework for Agentic Toolscited by 2
2026SABER: A Stealthy Agentic Black-Box Attack Framework for Vision-Language-Action Modelscited by 2

50 most cited of 109 All 109 (JSON)

Similar, not cited

Close in wording, no citation either way