mlsanityData to 14 Oct 2026

Scheduling Mixed RL Rollouts Beyond Prefix Locality

arXiv 2608.11152 ↗First posted 11 Aug 2026cs.DC cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Modern reinforcement learning (RL) post-training pipelines for large language models (LLMs) increasingly combine rollout workloads across multiple domains and feedback paradigms. Prefix-aware routing improves inference efficiency through cache reuse and load balancing, but it does not control how heterogeneous rollout sessions compete for KV-cache capacity. When reinforcement learning with verifiable rewards (RLVR), reinforcement learning from human feedback (RLHF), and agentic rollouts share an asynchronous inference service, their distinct sequence structures, interaction patterns, and KV-residency times create substantially different serving demands. Rollout scheduling must account for this heterogeneity without distorting the workload mixture specified by the trainer. We present MISA-T, a routing-layer admission policy for mixed rollout serving. MISA-T combines adaptive session admission, workload-aware KV-capacity allocation, and residency-time-aware KV accounting. In rollout-only ablations on Step3.7 and Qwen3.6-35B-A3B, MISA-T improves rollout throughput over a sweep-tuned cache-aware vLLM Router by 53.3% and 43.6%, respectively, while maintaining high prefix-cache hit rates. In a matched 50-iteration Step3.7 experiment, it increases rollout throughput by 35.6% and reduces mean iteration time by 22.8%, while keeping the consumed workload mixture close to the trainer target and achieving comparable task scores.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 23 of its 25 references that are in the map. First posted on arXiv 11 Aug 2026.

Builds on

23 of 25 references.

References · 23

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  5. 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoningcited by 250
  6. 2024InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Managementcited by 245
  7. 2026MiMo-V2-Flash Technical Reportcited by 191
  8. 2026Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolationcited by 184
  9. 2026Kimi K3: Open Frontier Intelligencecited by 138
  10. 2024ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inferencecited by 134
  11. 2026MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Trainingcited by 72
  12. 2024Preble: Efficient Distributed Prompt Scheduling for LLM Servingcited by 70
11 more
  1. 2026Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillationcited by 60
  2. 2026Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 42
  3. 2025MoDoMoDo: Multi-Domain Data Mixtures for Multimodal LLM Reinforcement Learningcited by 30
  4. 2026TraceLab: Characterizing Coding Agent Workloads for LLM Servingcited by 16
  5. 2025Locality-aware Fair Scheduling in LLM Servingcited by 15
  6. 2026DORA: A Scalable Asynchronous Reinforcement Learning System for Language Model Trainingcited by 14
  7. 2025UloRL:An Ultra-Long Output Reinforcement Learning Approach for Advancing Large Language Models' Reasoning Abilitiescited by 11
  8. 2026Multi-Rollout On-Policy Distillation via Peer Successes and Failurescited by 10
  9. 2025LLM Query Scheduling with Prefix Reuse and Latency Constraintscited by 9
  10. 2025AReaL-Hex: Accommodating Asynchronous RL Training over Heterogeneous GPUscited by 9
  11. 2026KAT-Coder-V2 Technical Reportcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Rollout Efficiency in Reinforcement Learning for Reasoning Large Language Models: A Taxonomy and Future Directionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025RollPacker: Mitigating Long-Tail Rollouts for Fast, Synchronous RL Post-Training
  2. 2026RolloutPipe: Overlapping Pipelined Rollout and Training in Disaggregated On-Policy LLM Reinforcement Learning
  3. 2026Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning
  4. 2026Contextual Rollout Bandits for Reinforcement Learning with Verifiable Rewards
  5. 2026Sparrow: Sparse Rollout for Stable and Efficient Long-context RL of Large Language Models
  6. 2026Libra: Efficient Resource Management for Agentic RL Post-Training
  7. 2025SPEC-RL: Accelerating On-Policy Reinforcement Learning with Speculative Rollouts
  8. 2026Spend Your Rollouts Where It Counts: Rollout Allocation for Group-Based RL Post-Training