mlsanityData to 14 Oct 2026

UbiMoE: A Ubiquitous Mixture-of-Experts Vision Transformer Accelerator With Hybrid Computation Pattern on FPGA

arXiv 2502.05602 ↗First posted 8 Feb 2025cs.ARcited by 4

On the map: Mixture-of-experts → Vision mixture-of-experts

Abstract (arXiv)

Compared to traditional Vision Transformers (ViT), Mixture-of-Experts Vision Transformers (MoE-ViT) are introduced to scale model size without a proportional increase in computational complexity, making them a new research focus. Given the high performance and reconfigurability, FPGA-based accelerators for MoE-ViT emerge, delivering substantial gains over general-purpose processors. However, existing accelerators often fall short of fully exploring the design space, leading to suboptimal trade-offs between resource utilization and performance. To overcome this problem, we introduce UbiMoE, a novel end-to-end FPGA accelerator tailored for MoE-ViT. Leveraging the unique computational and memory access patterns of MoE-ViTs, we develop a latency-optimized streaming attention kernel and a resource-efficient reusable linear kernel, effectively balancing performance and resource consumption. To further enhance design efficiency, we propose a two-stage heuristic search algorithm that optimally tunes hardware parameters for various FPGA resource constraints. Compared to state-of-the-art (SOTA) FPGA designs, UbiMoE achieves 1.34x and 3.35x throughput improvements for MoE-ViT on Xilinx ZCU102 and Alveo U280 platforms, respectively, while enhancing energy efficiency by 1.75x and 1.54x. Our implementation is available at https://github.com/DJ000011/UbiMoE.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts. Builds on 16 of its 26 references that are in the map. First posted on arXiv 8 Feb 2025.

Builds on

16 of 26 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Edge-MoE: Memory-Efficient Multi-Task Vision Transformer Architecture with Task-level Sparsity via Mixture-of-Experts30 May
  2. 2022M3ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-design26 Oct
  3. 2021Go Wider Instead of Deeper25 Jul
  4. 2021Scaling Vision with Sparse Mixture of Experts10 Jun

Other references · 12

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  3. 2023EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attentioncited by 172
  4. 2023Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inferencecited by 104
  5. 2021Sparse is Enough in Scaling Transformerscited by 86
  6. 2022DearKD: Data-Efficient Early Knowledge Distillation for Vision Transformerscited by 65
  7. 2024SHViT: Single-Head Vision Transformer with Memory Efficient Macro Designcited by 56
  8. 2022HeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformerscited by 46
  9. 2021FLAT: An Optimized Dataflow for Mitigating Attention Bottleneckscited by 39
  10. 2020ConfuciuX: Autonomous Hardware Resource Assignment for DNN Accelerators using Reinforcement Learningcited by 35
  11. 2020DNNExplorer: A Framework for Modeling and Exploring a Novel Paradigm of FPGA-based DNN Acceleratorcited by 18
  12. 2024MoNDE: Mixture of Near-Data Experts for Large-Scale Sparse Modelscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025CoQMoE: Co-Designed Quantization and Computation Orchestration for Mixture-of-Experts Vision Transformer on FPGAcited by 0
  2. 2026GraphLeap: Decoupling Graph Construction and Convolution for Vision GNN Acceleration on FPGAcited by 0
  3. 2026APEX: Adaptive Expert Prefetching for Memory-Efficient Edge MoE Inferencecited by 0
  4. 2026Recent Developments in Transformer Inference Deployment on FPGA Platforms: A Surveycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022VAQF: Fully Automatic Software-Hardware Co-Design Framework for Low-Bit Vision Transformer
  2. 2024Accelerating ViT Inference on FPGA through Static and Dynamic Pruning
  3. 2025LL-ViT: Edge Deployable Vision Transformers with Look Up Table Neurons
  4. 2022Auto-ViT-Acc: An FPGA-Aware Automatic Acceleration Framework for Vision Transformer with Mixed-Scheme Quantization
  5. 2024ViMoE: An Empirical Study of Designing Vision Mixture-of-Experts
  6. 2025Real Time FPGA Based Transformers & VLMs for Vision Tasks: SOTA Designs and Optimizations
  7. 2024Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Survey
  8. 2023A survey on efficient vision transformers: algorithms, techniques, and performance benchmarking