mlsanityData to 7 Oct 2026

RLAnything: Forge Environment, Policy, and Reward Model in Completely Dynamic RL System

arXiv 2602.02488 ↗First posted 2 Feb 2026cs.LG cs.AI cs.CLcited by 19

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

also in 1 more
Abstract (arXiv)

We propose RLAnything, a reinforcement learning framework that dynamically forges environment, policy, and reward models through closed-loop optimization, amplifying learning signals and strengthening the overall RL system for any LLM or agentic scenarios. Specifically, the policy is trained with integrated feedback from step-wise and outcome signals, while the reward model is jointly optimized via consistency feedback, which in turn further improves policy training. Moreover, our theory-motivated automatic environment adaptation improves training for both the reward and policy models by leveraging critic feedback from each, enabling learning from experience. Empirically, each added component consistently improves the overall system, and RLAnything yields substantial gains across various representative LLM and agentic tasks, boosting Qwen3-VL-8B-Thinking by 9.1% on OSWorld and Qwen2.5-7B-Instruct by 18.7% and 11.9% on AlfWorld and LiveBench, respectively. We also that optimized reward-model signals outperform outcomes that rely on human labels. Code: https://github.com/Gen-Verse/Open-AgentRL

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 52 of its 61 references that are in the map. First posted on arXiv 2 Feb 2026.

Builds on

52 of 61 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025AutoForge: Automated Environment Synthesis for Agentic Reinforcement Learning28 Dec
  2. 2025AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progress11 Nov
  3. 2025Don't Just Fine-tune the Agent, Tune the Environment11 Oct
  4. 2025Agentic Reinforced Policy Optimization26 Jul
  5. 2024EnvGen: Generating and Adapting Environments via LLMs for Training Embodied Agents18 Mar

Other references · 47

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2024Qwen2.5 Technical Reportcited by 4,713
  6. 2023Let's Verify Step by Stepcited by 3,598
  7. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  8. 2025Qwen3-VL Technical Reportcited by 2,534
  9. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  10. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  11. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  12. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
35 more
  1. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  2. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  3. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  4. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
  5. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 584
  6. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  7. 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 465
  8. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 450
  9. 2018TextWorld: A Learning Environment for Text-based Gamescited by 304
  10. 2025Inference-Time Scaling for Generalist Reward Modelingcited by 233
  11. 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learningcited by 205
  12. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 177
  13. 2025UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learningcited by 166
  14. 2025OpenCUA: Open Foundations for Computer-Use Agentscited by 151
  15. 2024Code Generation with AlphaCodium: From Prompt Engineering to Flow Engineeringcited by 100
  16. 2025S*: Test Time Scaling for Code Generationcited by 95
  17. 2025GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoningcited by 94
  18. 2025MAI-UI Technical Report: Real-World Centric Foundation GUI Agentscited by 88
  19. 2024Watch Every Step! LLM Agent Learning via Iterative Step-Level Process Refinementcited by 86
  20. 2025Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Modelscited by 80
  21. 2024Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learningcited by 63
  22. 2025DeepRetrieval: Hacking Real Search Engines and Retrievers with Large Language Models via Reinforcement Learningcited by 62
  23. 2025Co-Evolving LLM Coder and Unit Tester via Reinforcement Learningcited by 51
  24. 2025Towards General Agentic Intelligence via Environment Scalingcited by 51
  25. 2025ScoreFlow: Mastering LLM Agent Workflows via Score-based Preference Optimizationcited by 50
  26. 2023Enhancing Large Language Models in Coding Through Multi-Perspective Self-Consistencycited by 46
  27. 2026EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesiscited by 46
  28. 2025RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environmentscited by 43
  29. 2025Scaling Agent Learning via Experience Synthesiscited by 40
  30. 2025RL Tango: Reinforcing Generator and Verifier Together for Language Reasoningcited by 39
  31. 2025GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulatorscited by 34
  32. 2025UI-S1: Advancing GUI Automation via Semi-online Reinforcement Learningcited by 32
  33. 2025Efficient Multi-turn RL for GUI Agents via Decoupled Training and Adaptive Data Curationcited by 24
  34. 2025Generative Universal Verifier as Multimodal Meta-Reasonercited by 17
  35. 2024ACECode: A Reinforcement Learning Framework for Aligning Code Efficiency and Correctness in Code Language Modelscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 19.

19 papers.

Sort:

By area:

  1. 2026OpenClaw-RL: Train Any Agent Simply by Talkingcited by 83
  2. 2026Qwen-AgentWorld: Language World Models for General Agentscited by 22
  3. 2026SOD: Step-wise On-policy Distillation for Small Language Model Agentscited by 17
  4. 2026ToolCUA: Towards Optimal GUI-Tool Path Orchestration for Computer Use Agentscited by 10
  5. 2026Demystifying Reinforcement Learning for Long-Horizon Tool-Using Agents: A Comprehensive Recipecited by 6
  6. 2026SPADE: Self-Play in Adaptive Synthetic Executable Environmentscited by 6
  7. 2026ARCO: Adaptive Rubrics with Co-Evolution for Multi-Step LLM-Based Agentscited by 4
  8. 2026Discovery Foundation Models: Toward Open-Ended Discovery Intelligencecited by 3
  9. 2026ScienceBuddy: Recursive-in-Recursive Self-Improvement for Interactive Scientific Agentscited by 3
  10. 2026Co-Evolution of Policy and Internal Reward for Language Agentscited by 1
  11. 2026AstraFlow: Dataflow-Oriented Reinforcement Learning for Agentic LLMscited by 1
  12. 2026Verifiable Environments Are LEGO Bricks: Recursive Composition for Reasoning Generalizationcited by 1
7 more
  1. 2026Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Designcited by 1
  2. 2026Document Reconstruction Unlocks Scalable Long-Context RLVRcited by 0
  3. 2026Rethinking Agentic Reinforcement Learning In Large Language Modelscited by 0
  4. 2026ODYSSE: Episode-wise Policy Optimization for Personalized Agentic Reasoningcited by 0
  5. 2026From Scoring to Acting: Outcome-Verified Comparative Self-Distillation for LLM Agentscited by 0
  6. 2026Beyond Success and Failure: Length-Aware Contrastive Learning for GUI Agentscited by 0
  7. 2026V-Gym: Enhancing Agentic Visual Reasoning via Skill-Data Co-Evolutioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025One Framework to Rule Them All: Unifying RL-Based and RL-Free Methods in RLHF
  2. 2025A Technical Survey of Reinforcement Learning Techniques for Large Language Models
  3. 2026Complementary RL: Towards Efficient Experience-Driven Agent Learning
  4. 2025Refined Policy Distillation: From VLA Generalists to RL Experts
  5. 2026EnvRL: Learn from Environment Dynamics in Agentic Reinforcement Learning
  6. 2026ReSkill: Reconciling Skill Creation with Policy Optimization in Agentic RL
  7. 2025AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learning
  8. 2025RLAF: Reinforcement Learning from Automaton Feedback