Mixture of Predefined Experts: Maximizing Data Usage on Vertical Federated Learning

arXiv 2602.12708 ↗First posted 13 Feb 2026cs.LGcited by 0

On the map: Mixture-of-experts → Federated mixture-of-experts

Abstract (arXiv)

Vertical Federated Learning (VFL) has emerged as a critical paradigm for collaborative model training in privacy-sensitive domains such as finance and healthcare. However, most existing VFL frameworks rely on the idealized assumption of full sample alignment across participants, a premise that rarely holds in real-world scenarios. To bridge this gap, this work introduces Split-MoPE, a novel framework that integrates Split Learning with a specialized Mixture of Predefined Experts (MoPE) architecture. Unlike standard Mixture of Experts (MoE), where routing is learned dynamically, MoPE uses predefined experts to process specific data alignments, effectively maximizing data usage during both training and inference without requiring full sample overlap. By leveraging pretrained encoders for target data domains, Split-MoPE achieves state-of-the-art performance in a single communication round, significantly reducing the communication footprint compared to multi-round end-to-end training. Furthermore, unlike existing proposals that address sample misalignment, this novel architecture provides inherent robustness against malicious or noisy participants and offers per-sample interpretability by quantifying each collaborator's contribution to each prediction. Extensive evaluations on vision (CIFAR-10/100) and tabular (Breast Cancer Wisconsin) datasets demonstrate that Split-MoPE consistently outperforms state-of-the-art systems such as LASER and Vertical SplitNN, particularly in challenging scenarios with high data missingness.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Federated mixture-of-experts, one of 12 directions in Mixture-of-experts. Builds on 21 of its 34 references that are in the map. First posted on arXiv 13 Feb 2026.

Builds on

21 of 34 references.

References · 21

2015Deep Residual Learning for Image Recognitioncited by 51,809
2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
2019Advances and Open Problems in Federated Learningcited by 3,021
2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
2018Federated Optimization in Heterogeneous Networkscited by 2,971
2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
2020Adaptive Federated Optimizationcited by 1,036
2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
2021Scaling Vision with Sparse Mixture of Expertscited by 740
2018Split learning for health: Distributed deep learning without sharing raw patient datacited by 391

9 more

2022On the Representation Collapse of Sparse Mixture of Expertscited by 156
2024Harder Tasks Need More Experts: Dynamic Routing in MoE Modelscited by 71
2024MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Expertscited by 55
2020SplitNN-driven Vertical Partitioningcited by 42
2020FedCVT: Semi-supervised Vertical Federated Learning with Cross-view Trainingcited by 32
2023Approximating Two-Layer Feedforward Networks for Efficient Transformerscited by 27
2023Vertical Federated Knowledge Transfer via Representation Distillation for Healthcare Collaboration Networkscited by 19
2024Vertical Federated Learning with Missing Features During Training and Inferencecited by 6
2024Towards Active Participant Centric Vertical Federated Learning: Some Representations May Be All You Needcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way