mlsanityData to 7 Oct 2026

RISED: RubrIcs for agentic multi-environment Selection and sElf-Distillation

arXiv 2610.00979 ↗First posted 1 Oct 2026cs.AIcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Training a single LLM agent jointly across diverse interactive environments has attracted increasing attention as a route to generalist agents. Existing curriculum and data-selection strategies often allocate training at the environment level or prioritize local reward-based signals, without explicitly considering relationships between current rollouts across environments for prompt-group selection. Meanwhile, as environments are learned at different rates, all-failure and all-success rollout groups can coexist within a batch, leaving those data without group-relative reward signals. Both challenges highlight limitations of relying solely on scalar rewards in multi-environment RL: they provide limited information about cross-environment relationships and no within-group reward contrast when rewards are identical. This motivates richer textual feedback, such as rubrics describing rollout behaviours, to guide learning. Beyond rubrics' usage as reward, we repurpose rubrics to guide both online data selection and policy supervision. An LLM judge tags each rollout using a predefined rubric vocabulary shared across environments. The resulting profiles guide the selection of data that aligns with the overall behavioural composition of the mixed-environment batch while limiting overlap with already-selected data. Available positive rubrics (describing desired behaviours) provide privileged context for an on-policy self-distillation teacher, supplying additional token-level supervision, while negative rubrics (describing undesired behaviours) guide subsequent rollout generation away from recurring failure modes. Together, these components form RISED. Across model backbones, RISED achieves the highest mean pass rate across environments and ranks first or second in every individual environment. Rubric-based analysis of RISED can further characterize the behavioural changes accompanying these gains.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 56 of its 60 references that are in the map. First posted on arXiv 1 Oct 2026.

Builds on

56 of 60 references.

Closest ancestors · 8

Same direction, newest first

  1. 2026What and When to Distill: Selective Hindsight Distillation for Multi-Turn Agents19 May
  2. 2026Self-Distilled Agentic Reinforcement Learning14 May
  3. 2026Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agents12 Apr
  4. 2026CM2: Reinforcement Learning with Checklist Rewards for Multi-Turn and Multi-Step Agentic Tool Use12 Feb
  5. 2026Paying Less Generalization Tax: A Cross-Domain Generalization Study of RL Training for LLM Agents26 Jan
  6. 2025AgentRL: Scaling Agentic Reinforcement Learning with a Multi-Turn, Multi-Task Framework5 Oct
  7. 2025A Practitioner's Guide to Multi-turn Agentic Reinforcement Learning1 Oct
  8. 2025Reinforcement Learning for Long-Horizon Interactive LLM Agents3 Feb

Other references · 48

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  3. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  4. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  5. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  6. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
  7. 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelscited by 386
  8. 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 359
  9. 2025Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domainscited by 328
  10. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 319
  11. 2026Reinforcement Learning via Self-Distillationcited by 285
  12. 2026Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipecited by 262
36 more
  1. 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learningcited by 252
  2. 2022Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learntcited by 233
  3. 2025Llama-Nemotron: Efficient Reasoning Modelscited by 151
  4. 2015Online Batch Selection for Faster Training of Neural Networkscited by 121
  5. 2025Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rolloutscited by 108
  6. 2025Self-Evolving Curriculum for LLM Reasoningcited by 107
  7. 2026TIP: Token Importance in On-Policy Distillationcited by 90
  8. 2025Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 85
  9. 2025Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspectivecited by 75
  10. 2026Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillationcited by 53
  11. 2025Improving Data Efficiency for LLM Reinforcement Fine-tuning Through Difficulty-targeted Online Data Selection and Rollout Replaycited by 52
  12. 2026Expanding the Capabilities of Reinforcement Learning via Text Feedbackcited by 49
  13. 2026Not All Disagreement Is Learnable: Token Teachability in On-Policy Distillationcited by 42
  14. 2025Online Rubrics Elicitation from Pairwise Comparisonscited by 41
  15. 2025Prompt Curriculum Learning for Efficient LLM Post-Trainingcited by 40
  16. 2025VCRL: Variance-based Curriculum Reinforcement Learning for Large Language Modelscited by 35
  17. 2026Filter, Then Reweight: Rethinking Optimization Granularity in On-Policy Distillationcited by 31
  18. 2025MoDoMoDo: Multi-Domain Data Mixtures for Multimodal LLM Reinforcement Learningcited by 29
  19. 2026Open Rubric System: Scaling Reinforcement Learning with Pairwise Adaptive Rubriccited by 26
  20. 2026Trust Region On-Policy Distillationcited by 26
  21. 2026Reinforcing Chain-of-Thought Reasoning with Self-Evolving Rubricscited by 23
  22. 2026Unmasking On-Policy Distillation: Where It Helps, Where It Hurts, and Whycited by 21
  23. 2025Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuningcited by 14
  24. 2025Data-Efficient RLVR via Off-Policy Influence Guidancecited by 12
  25. 2025ActiveDPO: Active Direct Preference Optimization for Sample-Efficient Alignmentcited by 9
  26. 2026To Mix or To Merge: Toward Multi-Domain Reinforcement Learning for Large Language Modelscited by 9
  27. 2026RubiCap: Rubric-Guided Reinforcement Learning for Dense Image Captioningcited by 9
  28. 2026Not Every Rubric Teaches Equally: Policy-Aware Rubric Rewards for RLVRcited by 9
  29. 2025RuleReasoner: Reinforced Rule-based Reasoning via Domain-aware Dynamic Samplingcited by 6
  30. 2026Efficient RLVR Training via Weighted Mutual Information Data Selectioncited by 4
  31. 2026Transferability for General Reasoning: An Automated Curriculum for Multi-Domain RLVRcited by 4
  32. 2026CriPO: Enhancing Rubric-based RL via Self-Distillationcited by 4
  33. 2025Uncovering Intrinsic Capabilities: A Paradigm for Data Curation in Vision-Language Modelscited by 2
  34. 2026MagicAgent: Towards Generalized Agent Planningcited by 2
  35. 2026Rubrics as Privileged Information for Open-Ended Generationcited by 2
  36. 2026Beyond Simply Environment Scaling: Designing Effective Environment Distributions for Multimodal Agent Learningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ARCO: Adaptive Rubrics with Co-Evolution for Multi-Step LLM-Based Agents
  2. 2025AutoEnv: Automated Environments for Measuring Cross-Environment Agent Learning
  3. 2026From Holistic Evaluation to Structured Criteria: Rubrics Across the Evolving LLM Landscape
  4. 2026ARISE-RL: Agentic Rubric-Grounded Iterative Self-Evolution with Reinforcement Learning
  5. 2026ARISE: Adapting to Evolving Capability Gaps in Agentic Reinforcement Learning
  6. 2026Co-ReAct: Rubrics as Step-Level Collaborators for ReAct Agents
  7. 2026SEAL: Synergistic Co-Evolution of Agents and Learning Environments
  8. 2026State2State: Environment-Derived Mid-Training for LLM Agents