# SHEFL: Sparse Heterogeneous Ensemble Federated Learning with Group-Balanced Aggregation

arXiv: https://arxiv.org/abs/2508.08552 · HTML: https://mlsanity.com/paper/2508.08552/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 16 references that are in the map.
- First posted on arXiv 12 Aug 2025.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [Ensemble Distillation for Robust Model Fusion in Federated Learning](https://mlsanity.com/paper/2006.07242/) (2020)
- [HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients](https://mlsanity.com/paper/2010.01264/) (2020)
- [FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning](https://mlsanity.com/paper/2009.01974/) (2020)
- [FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extraction](https://mlsanity.com/paper/2212.01548/) (2022)
- [Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learning](https://mlsanity.com/paper/2204.12703/) (2022)
- [Fed-ensemble: Improving Generalization through Model Ensembling in Federated Learning](https://mlsanity.com/paper/2107.10663/) (2021)

## Abstract (arXiv)

System heterogeneity in federated learning (FL) requires training workloads and communication demands to be adapted to client resource capacities. Federated ensemble learning distributes multiple predictors across clients, but assigning one ensemble member to each client reduces the number of contributors to each member as the ensemble grows. We propose SHEFL, a global ensemble-based FL framework that uses the additional computation of high-power clients (HPCs) to train all ensemble members while low-power clients (LPCs) train one assigned member. SHEFL addresses increased HPC uplink demand and biased aggregation toward HPC updates through resource-aware sparse update allocation and group-balanced aggregation. Across four vision datasets, SHEFL improves over Fed-Ensemble (FedEns) in all data heterogeneity settings. Under a matched total uplink budget, SHEFL reaches the target accuracy in fewer rounds than FedEns and improves final accuracy by up to 14.65 points. Ablations further show that group-balanced aggregation improves accuracy and convergence under different resource allocations.