mlsanityData to 14 Oct 2026

RoutePack: Expert Placement and Attention-Aware Data Packing for MoE Reinforcement Learning

arXiv 2608.12146 ↗First posted 12 Aug 2026cs.DC cs.LGcited by 0

On the map: Mixture-of-experts → MoE in reinforcement learning and robotics

also in 1 more
Abstract (arXiv)

Training Mixture-of-Experts (MoE) models for reinforcement learning (RL) couples two load-balancing problems: sequence composition determines dense attention work in each data-parallel microbatch, while token routing determines sparse expert work on expert-parallel ranks. Optimizing either alone can shift the bottleneck to the other. In MoE RL, rollout-time routing replay exposes every sample's sequence length and layer-wise expert demand before its training step. We present RoutePack, a hierarchical planner that coordinates state-consistent, layer-wise expert rerouting with joint attention- and expert-aware data packing over an optimizer-step window. RoutePack first places experts independently at each MoE layer using aggregate routing demand. It then packs samples into the smallest certified, or best-known feasible, number of token-capped execution rows and optimizes their DP layout with a projected EDP-shard-aware objective. The objective combines a window-normalized linear-quadratic attention proxy with per-layer physical EP-rank peaks and minimizes the accumulated cost of the slowest EDP shard. Parallel population annealing searches fixed-row feasible layouts while preserving sample coverage, capacity, nonempty cells, equal microbatch counts, and communicator topology. State-consistent materialization preserves logical top-k routing and existing MoE kernels without microbatch-level expert replication. Across Ling-3.0-Tiny and Ling-3.0-Flash, expert rerouting improves mean trainer-measured token throughput by 3.80% and 10.50%, while routing-aware packing adds another 4.86% and 3.98%, respectively. Overall, RoutePack improves throughput by 8.85% and 14.89% over the baseline.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MoE in reinforcement learning and robotics, one of 12 directions in Mixture-of-experts. Builds on 14 of its 37 references that are in the map. First posted on arXiv 12 Aug 2026.

Builds on

14 of 37 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026Harnessing Routing Foresight for Micro-step-level MoE load balancing in RL Post-training10 Jun
  2. 2026ReLibra: Routing-Replay-Guided Load Balancing for MoE Training in Reinforcement Learning9 May
  3. 2025Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers13 Oct

Other references · 11

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  4. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  5. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
  6. 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoningcited by 250
  7. 2025Kimi Linear: An Expressive, Efficient Attention Architecturecited by 196
  8. 2025Semantic Parallelism: Redefining Efficient MoE Inference via Model-Data Co-Schedulingcited by 9
  9. 2026UltraEP: Unleash MoE Training and Inference on Rack-Scale Nodes with Near-Optimal Load Balancingcited by 9
  10. 2026Coordinated Scheduling for MoE LLM Servingcited by 2
  11. 2026Libra: Taming Attention Workload Skew in Long-Context LLM Training with Bounded Sequence Poolcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Expert-Space Exploration in MoE Reinforcement Learning
  2. 2025Towards Stable and Effective Reinforcement Learning for Mixture-of-Experts
  3. 2026Phase-Aware Mixture of Experts for Agentic Reinforcement Learning
  4. 2026Structuring MoE Expert Selection for Agentic Reinforcement Learning
  5. 2026MoE-GRPO: Optimizing Mixture-of-Experts via Reinforcement Learning in Vision-Language Models
  6. 2026MAPLE: MoE Adaptive Plug-and-play Layer-wise Expert allocation
  7. 2026Memory-Efficient Expert Routing for Distributed MoE Training
  8. 2026PR2: Predictive Routing Replay for MoE-Based LLM Reinforcement Learning