mlsanityData to 14 Oct 2026

Handling Data Heterogeneity via Architectural Design for Federated Visual Recognition

arXiv 2310.15165 ↗First posted 23 Oct 2023cs.CV cs.AI cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is a promising research paradigm that enables the collaborative training of machine learning models among various parties without the need for sensitive information exchange. Nonetheless, retaining data in individual clients introduces fundamental challenges to achieving performance on par with centrally trained models. Our study provides an extensive review of federated learning applied to visual recognition. It underscores the critical role of thoughtful architectural design choices in achieving optimal performance, a factor often neglected in the FL literature. Many existing FL solutions are tested on shallow or simple networks, which may not accurately reflect real-world applications. This practice restricts the transferability of research findings to large-scale visual recognition models. Through an in-depth analysis of diverse cutting-edge architectures such as convolutional neural networks, transformers, and MLP-mixers, we experimentally demonstrate that architectural choices can substantially enhance FL systems' performance, particularly when handling heterogeneous data. We study 19 visual recognition models from five different architectural families on four challenging FL datasets. We also re-investigate the inferior performance of convolution-based architectures in the FL setting and analyze the influence of normalization layers on the FL performance. Our findings emphasize the importance of architectural design for computer vision tasks in practical scenarios, effectively narrowing the performance gap between federated and centralized learning. Our source code is available at https://github.com/sarapieri/fed_het.git.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 54 of its 76 references that are in the map. First posted on arXiv 23 Oct 2023.

Builds on

54 of 76 references.

References · 54

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  6. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  7. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  8. 2022A ConvNet for the 2020scited by 3,579
  9. 2019Advances and Open Problems in Federated Learningcited by 3,046
  10. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  11. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,446
  12. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,228
38 more
  1. 2018Federated Learning with Non-IID Datacited by 1,108
  2. 2020Adaptive Federated Optimizationcited by 1,046
  3. 2017Deeper, Broader and Artier Domain Generalizationcited by 986
  4. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  5. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  6. 2018LEAF: A Benchmark for Federated Settingscited by 792
  7. 2021Model-Contrastive Federated Learningcited by 766
  8. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
  9. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 569
  10. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  11. 2017Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)cited by 549
  12. 2021MetaFormer Is Actually What You Need for Visioncited by 524
  13. 2021RepVGG: Making VGG-style ConvNets Great Againcited by 522
  14. 2022MaxViT: Multi-Axis Vision Transformercited by 476
  15. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 374
  16. 2021Pay Attention to MLPscited by 368
  17. 2021ResMLP: Feedforward networks for image classification with data-efficient trainingcited by 339
  18. 2020Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrievalcited by 309
  19. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 305
  20. 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 300
  21. 2021FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Spacecited by 287
  22. 2021Federated Learning for Internet of Things: A Comprehensive Surveycited by 252
  23. 2019BCN20000: Dermoscopic Lesions in the Wildcited by 225
  24. 2022Patches Are All You Need?cited by 201
  25. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  26. 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
  27. 2022MetaFormer Baselines for Visioncited by 132
  28. 2020Federated Visual Classification with Real-World Data Distributioncited by 128
  29. 2022FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settingscited by 125
  30. 2021Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learningcited by 111
  31. 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 109
  32. 2021An Image is Worth 16x16 Words, What is a Video Worth?cited by 87
  33. 2021Multi-institutional Collaborations for Improving Deep Learning-based Magnetic Resonance Image Reconstruction Using Federated Learningcited by 78
  34. 2023FedExP: Speeding Up Federated Averaging via Extrapolationcited by 69
  35. 2018On the insufficiency of existing momentum schemes for Stochastic Optimizationcited by 49
  36. 2018Effects of Degradations on Deep Neural Network Architecturescited by 48
  37. 2018On the Ineffectiveness of Variance Reduced Optimization for Deep Learningcited by 48
  38. 2019Federated Transfer Reinforcement Learning for Autonomous Drivingcited by 44

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 4; 2025: 3.

8 papers.

  1. 2024Handling Spatial-Temporal Data Heterogeneity for Federated Continual Learning via Tail Anchorcited by 9
  2. 2025FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learningcited by 4
  3. 2025Redefining non-IID Data in Federated Learning for Computer Vision Tasks: Migrating from Labels to Embeddings for Task-Specific Data Distributionscited by 3
  4. 2024Federated Multi-Task Learning on Non-IID Data Silos: An Experimental Studycited by 2
  5. 2023ARIA: On the Interaction Between Architectures, Initialization and Aggregation Methods for Federated Visual Classificationcited by 1
  6. 2024FedDTPT: Federated Discrete and Transferable Prompt Tuning for Black-Box Large Language Modelscited by 1
  7. 2024An Architecture Built for Federated Learning: Addressing Data Heterogeneity through Adaptive Normalization-Free Feature Recalibrationcited by 0
  8. 2025Single-Round Scalable Analytic Federated Learningcited by 0