mlsanityData to 13 Oct 2026

RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments

arXiv 2511.07317 ↗First posted 10 Nov 2025cs.CL cs.LGcited by 46

Not on the map. Why?

Abstract (arXiv)

We introduce Reinforcement Learning (RL) with Adaptive Verifiable Environments (RLVE), an approach using verifiable environments that procedurally generate problems and provide algorithmically verifiable rewards, to scale up RL for language models (LMs). RLVE enables each verifiable environment to dynamically adapt its problem difficulty distribution to the policy model's capabilities as training progresses. In contrast, static data distributions often lead to vanishing learning signals when problems are either too easy or too hard for the policy. To implement RLVE, we create RLVE-Gym, a large-scale suite of 400 verifiable environments carefully developed through manual environment engineering. Using RLVE-Gym, we show that environment scaling, i.e., expanding the collection of training environments, consistently improves generalizable reasoning capabilities. RLVE with joint training across all 400 environments in RLVE-Gym yields a 3.37% absolute average improvement across six reasoning benchmarks, starting from one of the strongest 1.5B reasoning LMs. By comparison, continuing this LM's original RL training yields only a 0.49% average absolute gain despite using over 3x more compute. We release our code publicly.

Key facts. Cited by 46 papers in the map (data through 13 Oct 2026). Builds on 47 of its 61 references that are in the map. First posted on arXiv 10 Nov 2025.

Builds on

47 of 61 references.

References · 47

  1. 2022Training language models to follow instructions with human feedbackcited by 15,856
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,091
  3. 2025Qwen3 Technical Reportcited by 8,510
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,247
  5. 2024Qwen2.5 Technical Reportcited by 4,840
  6. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,714
  7. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,729
  8. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,693
  9. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,951
  10. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,201
  11. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,117
  12. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,111
35 more
  1. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,020
  2. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 865
  3. 2025Kimi K2: Open Agentic Intelligencecited by 544
  4. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 507
  5. 2025Process Reinforcement through Implicit Rewardscited by 484
  6. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 375
  7. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 333
  8. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 330
  9. 2019Emergent Tool Use From Multi-Agent Autocurriculacited by 310
  10. 2025OpenThoughts: Data Recipes for Reasoning Modelscited by 296
  11. 2025DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoningcited by 268
  12. 2020Prioritized Level Replaycited by 192
  13. 2019Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutionscited by 186
  14. 2020Automatic Curriculum Learning For Deep RL: A Short Surveycited by 143
  15. 2025REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewardscited by 126
  16. 2025BIG-Bench Extra Hardcited by 124
  17. 2024On Memorization of Large Language Models in Logical Reasoningcited by 115
  18. 2025Self-Evolving Curriculum for LLM Reasoningcited by 111
  19. 2025The Art of Scaling Reinforcement Learning Compute for LLMscited by 96
  20. 2025SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learningcited by 95
  21. 2025Efficient Reinforcement Finetuning via Adaptive Curriculum Learningcited by 91
  22. 2025SPICE: Self-Play In Corpus Environments Improves Reasoningcited by 80
  23. 2025What Makes a Reward Model a Good Teacher? An Optimization Perspectivecited by 70
  24. 2025Enigmata: Scaling Logical Reasoning in Large Language Models with Synthetic Verifiable Puzzlescited by 53
  25. 2025OMEGA: Can LLMs Reason Outside the Box in Math? Evaluating Exploratory, Compositional, and Transformative Generalizationcited by 49
  26. 2025SynLogic: Synthesizing Verifiable Reasoning Data at Scale for Learning Logical Reasoning and Beyondcited by 48
  27. 2025Prompt Curriculum Learning for Efficient LLM Post-Trainingcited by 42
  28. 2025APRIL: Active Partial Rollouts in Reinforcement Learning to Tame Long-tail Generationcited by 27
  29. 2023Vanishing Gradients in Reinforcement Finetuning of Language Modelscited by 26
  30. 2025EvalTree: Profiling Language Model Weaknesses via Hierarchical Capability Treescited by 26
  31. 2024DataEnvGym: Data Generation Agents in Teacher Environments with Student Feedbackcited by 24
  32. 2025InternBootcamp: Boosting LLM Reasoning with Verifiable Task Scalingcited by 19
  33. 2025Beyond 'Aha!': Toward Systematic Meta-Abilities Alignment in Large Reasoning Modelscited by 18
  34. 2024The Need for a Big World Simulator: A Scientific Challenge for Continual Learningcited by 5
  35. 2025Teaching LLM to Reason: Reinforcement Learning from Algorithmic Problems without Codecited by 3

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 4; 2026: 42.

46 papers.

Sort:

By area:

  1. 2025ThetaEvolve: Test-time Learning on Open Problemscited by 57
  2. 2025From Word to World: Can Large Language Models be Implicit Text-based World Models?cited by 42
  3. 2025Cognitive Foundations for Reasoning and Their Manifestation in LLMscited by 26
  4. 2026CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agentscited by 22
  5. 2026RLAnything: Forge Environment, Policy, and Reward Model in Completely Dynamic RL Systemcited by 20
  6. 2026Rethinking On-Policy Distillation of Large Language Models II: One Training Examplecited by 18
  7. 2025Environment Scaling for Interactive Agentic Experience Collection: A Surveycited by 13
  8. 2026EvoTrainer: Co-Evolving LLM Policies and Training Harnesses for Autonomous Agentic Reinforcement Learningcited by 10
  9. 2026Golden Goose: A Simple Trick to Synthesize Unlimited RLVR Tasks from Unverifiable Internet Textcited by 9
  10. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  11. 2026SETA: Scaling Environments for Terminal Agentscited by 8
  12. 2026Frontis-MA1: Training an AI4AI Model towards Recursive Self-Improvement in Machine Learning Engineeringcited by 8
34 more
  1. 2026SPADE: Self-Play in Adaptive Synthetic Executable Environmentscited by 8
  2. 2026SCALER:Synthetic Scalable Adaptive Learning Environment for Reasoningcited by 7
  3. 2026Do We Need Adam? Surprisingly Strong and Sparse Reinforcement Learning with SGD in LLMscited by 6
  4. 2026IsoCompute Playbook: Optimally Scaling Sampling Compute for LLM RLcited by 6
  5. 2026LongCoT: Benchmarking Long-Horizon Chain-of-Thought Reasoningcited by 6
  6. 2026Learning to Build the Environment: Self-Evolving Reasoning RL via Verifiable Environment Synthesiscited by 6
  7. 2026Composition-RL: Compose Your Verifiable Prompts for Reinforcement Learning of Large Language Modelscited by 5
  8. 2026ZAYA1-8B Technical Reportcited by 4
  9. 2026Training Reasoning Models on Saturated Problems via Failure-Prefix Conditioningcited by 3
  10. 2026Data Science and Technology Towards AGI Part I: Tiered Data Managementcited by 3
  11. 2026Gym-V: A Unified Vision Environment System for Agentic Vision Researchcited by 3
  12. 2026Learning to Reason with Curriculum I: Provable Benefits of Autocurriculumcited by 3
  13. 2026TRON: Targeted Rule-Verifiable Online Environments for Visual Reasoning RLcited by 3
  14. 2026Code2Math: Can Your Code Agent Effectively Evolve Math Problems Through Exploration?cited by 2
  15. 2026S3-R1: Learning to Retrieve and Answer Step-by-Step with Synthetic Datacited by 2
  16. 2026DocArena: Turning Raw Documents into Controllable Training Environments for Document Search Agentscited by 2
  17. 2026Learning to Reason with Curriculum II: Compositional Generalizationcited by 2
  18. 2026Beyond Simply Environment Scaling: Designing Effective Environment Distributions for Multimodal Agent Learningcited by 2
  19. 2026SFT Conflicts, RL Coexists: A Theoretical and Empirical Analysis of Multi-Task Learning for LLMscited by 2
  20. 2026GIFT: Games as Informal Training for Generalizable LLMscited by 1
  21. 2026Verifiable Environments Are LEGO Bricks: Recursive Composition for Reasoning Generalizationcited by 1
  22. 2026CircuitLens: Reasoning Circuits as Data Selection Signals for Reinforcement Learning with Verifiable Rewardscited by 1
  23. 2026Document Reconstruction Unlocks Scalable Long-Context RLVRcited by 0
  24. 2026Adaptive Problem Generation via Symbolic Representationscited by 0
  25. 2026ShapeCodeBench: A Renewable Benchmark for Perception-to-Program Reconstruction of Synthetic Shape Scenescited by 0
  26. 2026Reinforcement Learning without Ground-Truth Solutions can Improve LLMscited by 0
  27. 2026Verifiable Hidden Dynamics Play: Generating Agentic RL Environments from Solved Mechanismscited by 0
  28. 2026Breaking the Environment Wall: Evolving LLM Agent Environments for Recursive Self-Improvementcited by 0
  29. 2026Selecting Diverse SFT Traces Improves Post-RL Generalizationcited by 0
  30. 2026RAISE: Reinforcing Access Control Policy Synthesis in LLMs via Symbolic Evaluationcited by 0
  31. 2026Learning from Teacher Continuations at Student Statescited by 0
  32. 2026WEFT: Scaling Tool-Use Post-Training for General-Purpose Agentscited by 0
  33. 2026NeutronGym: Physics-Graded Neutron Instrument Design for LLM Agentscited by 0
  34. 2026PyINE: A Framework for Scalable Elicitation and Oversight via Code Executioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Resource-Efficient Reinforcement for Reasoning Large Language Models via Dynamic One-Shot Policy Refinement
  2. 2025Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection
  3. 2026ReSyn: Autonomously Scaling Synthetic Environments for Reasoning Models
  4. 2025A Survey of Reinforcement Learning for Large Reasoning Models
  5. 2025Beyond Verifiable Rewards: Scaling Reinforcement Learning for Language Models to Unverifiable Data
  6. 2026DARL: Encouraging Diverse Answers for General Reasoning without Verifiers
  7. 2025Extending RLVR to Open-Ended Tasks via Verifiable Multiple-Choice Reformulation
  8. 2026Linear Dynamics in the RLVR Training of Large Language Models