mlsanityData to 14 Oct 2026

FaaSMoE: A Serverless Framework for Multi-Tenant Mixture-of-Experts Serving

arXiv 2604.26881 ↗First posted 29 Apr 2026cs.DC cs.LGcited by 0

On the map: Mixture-of-experts → Efficient MoE training and inference systems

Abstract (arXiv)

Mixture-of-Experts (MoE) models offer high capacity with efficient inference cost by activating a small subset of expert models per input. However, deploying MoE models requires all experts to reside in memory, creating a gap between the resource used by activated experts and the provisioned resources. This underutilization is further pronounced in multi-tenant scenarios. In this paper, we propose FaaSMoE, a multi-tenant MoE serving architecture built on Function-as-a-Service (FaaS) platforms. FaaSMoE decouples the control and execution planes of MoE by deploying experts as stateless FaaS functions, enabling on-demand and scale-to-zero expert invocation across tenants. FaaSMoE further supports configurable expert granularity within functions, trading off per-expert elasticity for reduced invocation overhead. We implement a prototype with an open-source edge-oriented FaaS platform and evaluate it using Qwen1.5-moe-2.7B under multi-tenant workloads. Compared to a full-model baseline, FaaSMoE uses less than one third of the resources, demonstrating a practical and resource-efficient path towards scalable MoE serving in a multi-tenant environment.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts. Builds on 12 of its 27 references that are in the map. First posted on arXiv 29 Apr 2026.

Builds on

12 of 27 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Expert-as-a-Service: Towards Efficient, Scalable, and Robust Large-scale MoE Serving22 Sep
  2. 2025Optimizing Distributed Deployment of Mixture-of-Experts Model Inference in Serverless Computing9 Jan
  3. 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale14 Jan

Other references · 9

  1. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  2. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  3. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
  4. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 867
  5. 2021ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learningcited by 256
  6. 2020Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Providercited by 66
  7. 2023Lotus: Serverless In-Transit Data Processing for Edge-based Pub/Subcited by 1
  8. 2023The Night Shift: Understanding Performance Variability of Cloud Serverless Platformscited by 1
  9. 2025Towards a Testbed for Scalable FaaS Platformscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Remoe: Towards Efficient and Low-Cost MoE Inference in Serverless Computing
  2. 2026MoEless: Efficient MoE LLM Serving via Serverless Computing
  3. 2026Beyond Uniform Experts: Cost-Aware Expert Execution for Efficient Multi-Device MoE Inference
  4. 2026BrownoutMoE: Structure-Aware Expert Grouping for Efficient and Accurate LLM Web-based Services
  5. 2024HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inference
  6. 2026EasyBalance: Cross-Layer Load Balancing in Distributed MoE Inference
  7. 2025Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
  8. 2025ElasticMoE: An Efficient Auto Scaling Method for Mixture-of-Experts Models