mlsanityData to 7 Oct 2026

Temporally Extended Mixture-of-Experts Models

arXiv 2604.20156 ↗First posted 22 Apr 2026cs.LGcited by 2

On the map: Mixture-of-experts → MoE in reinforcement learning and robotics

Abstract (arXiv)

Mixture-of-Experts models, now popular for scaling capacity at fixed inference speed, switch experts at nearly every token. Once a model outgrows available GPU memory, this churn can render optimizations like offloading and pre-fetching ineffective. We make the case that the options framework in reinforcement learning is a perfect match to tackle this problem, and argue for temporally extended mixture-of-experts layers. Building on the option-critic framework with deliberation costs, we add a controller to each layer that learns when to switch expert sets and which to load. By applying this to gpt-oss-20b with low-rank adapters and a self-distillation reward, our method reduces switch rates from over 50% to below 5% while retaining up to 90% of base-model accuracy on MATH, MMLU, and MMMLU. This shows that even existing pre-trained models can be converted to temporally extended MoEs with lightweight training, with the deliberation cost allowing model trainers to trade off switching rates against capability. We hope this opens a principled path, grounded in the options framework, for memory-efficient serving and continual learning in ever-growing MoE models.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in MoE in reinforcement learning and robotics, one of 12 directions in Mixture-of-experts. Builds on 26 of its 33 references that are in the map. First posted on arXiv 22 Apr 2026.

Builds on

26 of 33 references.

References · 26

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  5. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  6. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  7. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  8. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
  9. 2016The Option-Critic Architecturecited by 433
  10. 2022Unified Scaling Laws for Routed Language Modelscited by 249
  11. 2024A Survey on Mixture of Experts in Large Language Modelscited by 233
  12. 2024Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Modelscited by 140
14 more
  1. 2022ERNIE-ViLG 2.0: Improving Text-to-Image Diffusion Model with Knowledge-Enhanced Mixture-of-Denoising-Expertscited by 110
  2. 2023Fast Inference of Mixture-of-Experts Language Models with Offloadingcited by 88
  3. 2024Mixture of A Million Expertscited by 70
  4. 2024MoE-Pruner: Pruning Mixture-of-Experts Large Language Model using the Hints from Its Routercited by 58
  5. 2017When Waiting is not an Option : Learning Options with a Deliberation Costcited by 57
  6. 2024Efficient Expert Pruning for Sparse Mixture-of-Experts Language Models: Enhancing Performance and Reducing Inference Costscited by 51
  7. 2024MoE-Infinity: Efficient MoE Inference on Personal Machines with Sparsity-Aware Expert Cachecited by 48
  8. 2024SEER-MoE: Sparse Expert Efficiency through Regularization for Mixture-of-Expertscited by 40
  9. 2024ProMoE: Fast MoE-based LLM Serving using Proactive Cachingcited by 38
  10. 2021Flexible Option Learningcited by 24
  11. 2024MA-RLHF: Reinforcement Learning from Human Feedback with Macro Actionscited by 15
  12. 2025eMoE: Task-aware Memory Efficient Mixture-of-Experts-Based (MoE) Model Inferencecited by 11
  13. 2025Enhancing Math Reasoning in Small-sized LLMs via Preview Difficulty-Aware Interventioncited by 3
  14. 2025Emergent temporal abstractions in autoregressive models enable hierarchical reinforcement learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026EMO: Pretraining Mixture of Experts for Emergent Modularitycited by 3
  2. 2026Cache-Aware Joint Router Adaptation for Memory-Efficient MoE Inferencecited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2026MELINOE: Fine-Tuning Enables Memory-Efficient Inference for Mixture-of-Experts Models
  2. 2024Mixtures of Experts Unlock Parameter Scaling for Deep RL
  3. 2026Flattening Every Memory Peak in Long-Context Mixture-of-Experts Training
  4. 2025Towards Stable and Effective Reinforcement Learning for Mixture-of-Experts
  5. 2026Moebius: Serving Mixture-of-Expert Models with Seamless Runtime Parallelism Switch
  6. 2026Optimal Expert-Attention Allocation in Mixture-of-Experts: A Scalable Law for Dynamic Model Design
  7. 2024ExpertFlow: Efficient Mixture-of-Experts Inference via Predictive Expert Caching and Token Scheduling
  8. 2026Mixture-of-Parallelisms: Towards Memory-Efficient Training Stack for Mixture-of-Experts Models