mlsanityData to 7 Oct 2026

HeteroSwitch: Characterizing and Taming System-Induced Data Heterogeneity in Federated Learning

arXiv 2403.04207 ↗First posted 7 Mar 2024cs.LG cs.DCcited by 1

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is a practical approach to train deep learning models collaboratively across user-end devices, protecting user privacy by retaining raw data on-device. In FL, participating user-end devices are highly fragmented in terms of hardware and software configurations. Such fragmentation introduces a new type of data heterogeneity in FL, namely textitsystem-induced data heterogeneity, as each device generates distinct data depending on its hardware and software configurations. In this paper, we first characterize the impact of system-induced data heterogeneity on FL model performance. We collect a dataset using heterogeneous devices with variations across vendors and performance tiers. By using this dataset, we demonstrate that textitsystem-induced data heterogeneity negatively impacts accuracy, and deteriorates fairness and domain generalization problems in FL. To address these challenges, we propose HeteroSwitch, which adaptively adopts generalization techniques (i.e., ISP transformation and SWAD) depending on the level of bias caused by varying HW and SW configurations. In our evaluation with a realistic FL dataset (FLAIR), HeteroSwitch reduces the variance of averaged precision by 6.3% across device types.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 33 of its 61 references that are in the map. First posted on arXiv 7 Mar 2024.

Builds on

33 of 61 references.

References · 33

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2019Searching for MobileNetV3cited by 1,984
  5. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  6. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  7. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  8. 2018Federated Learning with Non-IID Datacited by 1,099
  9. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
  10. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 762
  11. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
  12. 2019Agnostic Federated Learningcited by 451
21 more
  1. 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 431
  2. 2019Domain Generalization via Model-Agnostic Learning of Semantic Featurescited by 405
  3. 2019Fair Resource Allocation in Federated Learningcited by 389
  4. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
  5. 2020Environment Inference for Invariant Learningcited by 306
  6. 2016Understanding How Image Quality Affects Deep Neural Networkscited by 118
  7. 2021Papaya: Practical, Private, and Scalable Federated Learningcited by 89
  8. 2015A Deeper Look at Dataset Biascited by 86
  9. 2021Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Developmentcited by 85
  10. 2020Batch Normalization Embeddings for Deep Domain Generalizationcited by 57
  11. 2021Towards Fairness-Aware Federated Learningcited by 57
  12. 2020Training Speech Recognition Models with Federated Learning: A Quality/Cost Frameworkcited by 46
  13. 2021Node Selection Toward Faster Convergence for Federated Learning on Non-IID Datacited by 39
  14. 2021AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learningcited by 39
  15. 2017Reconfiguring the Imaging Pipeline for Computer Visioncited by 32
  16. 2022FLAIR: Federated Learning Annotated Image Repositorycited by 22
  17. 2022Towards Fair Federated Recommendation Learning: Characterizing the Inter-Dependence of System and Data Heterogeneitycited by 17
  18. 2019Neural Network Generalization: The impact of camera parameterscited by 14
  19. 2016Image Prediction for Limited-angle Tomography via Deep Learning with Convolutional Neural Networkcited by 9
  20. 2020Characterizing and Taming Model Instability Across Edge Devicescited by 7
  21. 2022FEL: High Capacity Learning for Recommendation and Ranking via Federated Ensemble Learningcited by 4

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Federated Learning Clients Clustering with Adaptation to Data Driftscited by 3