Dynamic Mixture-of-Experts for Visual Autoregressive Model
On the map: Mixture-of-experts → Vision mixture-of-experts
Abstract (arXiv)
Visual Autoregressive Models (VAR) offer efficient and high-quality image generation but suffer from computational redundancy due to repeated Transformer calls at increasing resolutions. We introduce a dynamic Mixture-of-Experts router integrated into VAR. The new architecture allows to trade compute for quality through scale-aware thresholding. This thresholding strategy balances expert selection based on token complexity and resolution, without requiring additional training. As a result, we achieve 20% fewer FLOPs, 11% faster inference and match the image quality achieved by the dense baseline.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts. Builds on 10 of its 14 references that are in the map. First posted on arXiv 8 Oct 2025.
Builds on
10 of 14 references.
References · 10
- 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
- 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
- 2016Adaptive Computation Time for Recurrent Neural Networkscited by 506
- 2025Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Stepscited by 234
- 2021MoEfication: Transformer Feed-forward Layers are Mixtures of Expertscited by 152
- 2023ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Modelscited by 94
- 2024Collaborative Decoding Makes Visual Auto-Regressive Modeling Efficientcited by 33
- 2025Memory-Efficient Visual Autoregressive Modeling with Scale-Aware KV Cache Compressioncited by 22
- 2024LiteVAR: Compressing Visual Autoregressive Modelling with Efficient Attention and Quantizationcited by 17
- 2023Exploiting Activation Sparsity with Dense to Dynamic-k Mixture-of-Experts Conversioncited by 16
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026MEPA: Multi-Scale Representation Alignment for Visual Autoregressive Modeling with Mixture of Experts
- 2025ActVAR: Activating Mixtures of Weights and Tokens for Efficient Visual Autoregressive Generation
- 2025Universal Approximation of Visual Autoregressive Transformers
- 2026Inference-Time Scaling for Visual AutoRegressive modeling by Searching Representative Samples
- 2024M-VAR: Decoupled Scale-wise Autoregressive Modeling for High-Quality Image Generation
- 2026Logit Refiner: Improving Visual Autoregressive Models via Intra-Scale Dependency Modeling
- 2025FlexVAR: Flexible Visual Autoregressive Modeling without Residual Prediction
- 2024Mixture of Nested Experts: Adaptive Processing of Visual Tokens