Effective Approaches to Batch Parallelization for Dynamic Neural Network Architectures
On the map: Mixture-of-experts → MoE theory and gating functions
Abstract (arXiv)
We present a simple dynamic batching approach applicable to a large class of dynamic architectures that consistently yields speedups of over 10x. We provide performance bounds when the architecture is not known a priori and a stronger bound in the special case where the architecture is a predetermined balanced tree. We evaluate our approach on Johnson et al.'s recent visual question answering (VQA) result of his CLEVR dataset by Inferring and Executing Programs (IEP). We also evaluate on sparsely gated mixture of experts layers and achieve speedups of up to 1000x over the naive implementation.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MoE theory and gating functions, one of 12 directions in Mixture-of-experts. First posted on arXiv 8 Jul 2017.
Builds on
No reference list yet.
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026DynaNDE: Dynamic Near-Data Expert Scheduling for Batched MoE Inference
- 2022Optimizing Mixture of Experts using Dynamic Recompilations
- 2022A Survey on Dynamic Neural Networks for Natural Language Processing
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
- 2024Memory Layers at Scale
- 2026Dynamic Expert Sharing: Decoupling Memory from Parallelism in Mixture-of-Experts Diffusion LLMs
- 2026DynaMoE: Dynamic Token-Level Expert Activation with Layer-Wise Adaptive Capacity for Mixture-of-Experts Neural Networks
- 2025Balancing Computation Load and Representation Expressivity in Parallel Hybrid Neural Networks