mlsanityData to 14 Oct 2026

AirMoE: Statistic-Augmented Over-the-Air MoE for Collaborative Intelligence

arXiv 2607.16562 ↗First posted 18 Jul 2026cs.DCcited by 0

On the map: Mixture-of-experts → Efficient MoE training and inference systems

Abstract (arXiv)

Mixture of Experts (MoE) are increasingly deployed over wireless cloud-edge networks, as a single edge device lacks sufficient resources to host large-scale models locally. In this distributed architecture, a cloud-hosted pretrained Large Model (LM) acts as a shared backbone for latent feature extraction, while heterogeneous experts deployed across distributed, wirelessly-connected clients collaboratively form the task head. However, deploying MoE over wireless links exposes two coupled bottlenecks. On the one hand, routing which clients to activate generally overloads bandwidth-limited uplinks due to required raw feature transmission. On the other hand, aggregating the activated experts' outputs over wireless links is hindered by channel noise and poor scalability. To break these bottlenecks, we propose a statistic-augmented over-the-air MoE (AirMoE) paradigm. Specifically, on the routing side, each client queries its local Feature Retrieval Library (FRL) with a cloud-broadcast compact query, retrieves a prototype-induced statistic, and reports it digitally to the cloud, drastically reducing uplink traffic; the cloud then selects the most relevant clients by aligning these statistics with the LM-extracted features via Jensen--Shannon (JS) divergence. On the aggregating side, selected experts simultaneously transmit their outputs over the multiple-access channel, which physically computes the reweighted sum via waveform superposition, with reweighting coefficients realized through channel-aware power control. The two mechanisms are thus decoupled both algorithmically and physically. We further provide theoretical analyses on convergence and iteration complexity. Taking semantic segmentation task as an example, extensive experiments demonstrate that AirMoE outperforms MoE baselines and single-model competitors. Ablations further confirm the effectiveness of each incorporated component.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts. Builds on 47 of its 60 references that are in the map. First posted on arXiv 18 Jul 2026.

Builds on

47 of 60 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024WDMoE: Wireless Distributed Large Language Models with Mixture of Experts6 May
  2. 2022Accelerating Distributed MoE Training and Inference with Lina31 Oct
  3. 2022Tutel: Adaptive Mixture-of-Experts at Scale7 Jun

Other references · 44

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  5. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  6. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  7. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  8. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,161
  9. 2024Mixtral of Expertscited by 1,823
  10. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  11. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,487
  12. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
32 more
  1. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 867
  2. 2021Scaling Vision with Sparse Mixture of Expertscited by 747
  3. 2022Mixture-of-Experts with Expert Choice Routingcited by 623
  4. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 596
  5. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 324
  6. 2021BASE Layers: Simplifying Training of Large, Sparse Modelscited by 320
  7. 2020BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentationcited by 294
  8. 2023From Sparse to Soft Mixtures of Expertscited by 248
  9. 2021Hash Layers For Large Sparse Modelscited by 237
  10. 2018Federated Learning via Over-the-Air Computationcited by 219
  11. 2018Broadband Analog Aggregation for Low-Latency Federated Edge Learning (Extended Version)cited by 191
  12. 2018The ApolloScape Open Dataset for Autonomous Driving and its Applicationcited by 190
  13. 2021Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challengescited by 115
  14. 2022StableMoE: Stable Routing Strategy for Mixture of Expertscited by 100
  15. 2022M3ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-designcited by 98
  16. 2025A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applicationscited by 91
  17. 2022TopFormer: Token Pyramid Transformer for Mobile Semantic Segmentationcited by 82
  18. 2018JointDNN: An Efficient Training and Inference Engine for Intelligent Mobile Cloud Computing Servicescited by 80
  19. 2019BottleNet: A Deep Learning Architecture for Intelligent Mobile Cloud Computing Servicescited by 68
  20. 2020Communication-Computation Trade-Off in Resource-Constrained Edge Inferencecited by 60
  21. 2021Optimized Power Control Design for Over-the-Air Federated Edge Learningcited by 48
  22. 2023Domain Adaptive and Generalizable Network Architectures and Training Strategies for Semantic Image Segmentationcited by 26
  23. 2024pFedLVM: A Large Vision Model (LVM)-Driven and Latent Feature-Based Personalized Federated Learning Framework in Autonomous Drivingcited by 16
  24. 2021AttaNet: Attention-Augmented Network for Fast and Accurate Scene Parsingcited by 14
  25. 2024FedRC: A Rapid-Converged Hierarchical Federated Learning Framework in Street Scene Semantic Understandingcited by 8
  26. 2024Fast-Convergent and Communication-Alleviated Heterogeneous Hierarchical Federated Learning in Autonomous Drivingcited by 8
  27. 2023Communication Resources Constrained Hierarchical Federated Learning for End-to-End Autonomous Drivingcited by 7
  28. 2024Theory of Mixture-of-Experts for Mobile Edge Computingcited by 5
  29. 2025Mixture-of-Experts for Distributed Edge Computing with Channel-Aware Gating Functioncited by 5
  30. 2025FedEMA: Federated Exponential Moving Averaging with Negative Entropy Regularizer in Autonomous Drivingcited by 5
  31. 2025A Unified Distributed Algorithm for Hybrid Near-Far Field Activity Detection in Cell-Free Massive MIMOcited by 2
  32. 2025Fast AI Model Partition for Split Learning over Edge Networkscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025EC2MoE: Adaptive End-Cloud Pipeline Collaboration Enabling Scalable Mixture-of-Experts Inference
  2. 2024Shortcut-connected Expert Parallelism for Accelerating Mixture-of-Experts
  3. 2025Symphony-MoE: Harmonizing Disparate Pre-trained Models into a Coherent Mixture-of-Experts
  4. 2023Efficient Deweather Mixture-of-Experts with Uncertainty-aware Feature-wise Linear Modulation
  5. 2022MoESys: A Distributed and Efficient Mixture-of-Experts Training and Inference System for Internet Services
  6. 2026Scaling Multi-Node Mixture-of-Experts Inference Using Expert Activation Patterns
  7. 2026LatentMoE: Toward Optimal Accuracy per FLOP and Parameter in Mixture of Experts
  8. 2026HFedMoE: Resource-aware Heterogeneous Federated Learning with Mixture-of-Experts