mlsanityData to 7 Oct 2026

Federated Learning Over Images: Vertical Decompositions and Pre-Trained Backbones Are Difficult to Beat

arXiv 2309.03237 ↗First posted 6 Sep 2023cs.LG cs.IT math.ITcited by 4

Not on the map. Why?

Abstract (arXiv)

We carefully evaluate a number of algorithms for learning in a federated environment, and test their utility for a variety of image classification tasks. We consider many issues that have not been adequately considered before: whether learning over data sets that do not have diverse sets of images affects the results; whether to use a pre-trained feature extraction "backbone"; how to evaluate learner performance (we argue that classification accuracy is not enough), among others. Overall, across a wide variety of settings, we find that vertically decomposing a neural network seems to give the best results, and outperforms more standard reconciliation-used methods.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 40 of its 55 references that are in the map. First posted on arXiv 6 Sep 2023.

Builds on

40 of 55 references.

References · 40

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  6. 2016Densely Connected Convolutional Networkscited by 6,691
  7. 2019Advances and Open Problems in Federated Learningcited by 3,021
  8. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  9. 2013Describing Textures in the Wildcited by 2,110
  10. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  11. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  12. 2018Federated Learning with Non-IID Datacited by 1,099
28 more
  1. 2020Adaptive Federated Optimizationcited by 1,036
  2. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  3. 2019Towards Federated Learning at Scale: System Designcited by 826
  4. 2021Model-Contrastive Federated Learningcited by 762
  5. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  6. 2015An Introduction to Convolutional Neural Networkscited by 510
  7. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  8. 2018Don't Use Large Mini-Batches, Use Local SGDcited by 201
  9. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  10. 2019Privacy-preserving Federated Brain Tumour Segmentationcited by 169
  11. 2020Federated Visual Classification with Real-World Data Distributioncited by 126
  12. 2020FedVision: An Online Visual Object Detection Platform Powered by Federated Learningcited by 121
  13. 2018Learning to Navigate for Fine-grained Classificationcited by 103
  14. 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
  15. 2017On Pre-Trained Image Features and Synthetic Images for Deep Learningcited by 51
  16. 2020Synthetic Learning: Learn From Distributed Asynchronized Discriminator GAN Without Sharing Medical Image Datacited by 38
  17. 2021Few-shot Image Classification: Just Use a Library of Pre-trained Feature Extractors and a Simple Classifiercited by 32
  18. 2021FedCV: A Federated Learning Framework for Diverse Computer Vision Taskscited by 32
  19. 2016Parallel SGD: When does averaging help?cited by 30
  20. 2019Distributed Learning of Deep Neural Networks using Independent Subnet Trainingcited by 20
  21. 2021ResIST: Layer-Wise Decomposition of ResNets for Distributed Trainingcited by 18
  22. 2020Fed-Sim: Federated Simulation for Medical Imagingcited by 14
  23. 2021Partial Variable Training for Efficient On-Device Federated Learningcited by 14
  24. 2021On the Convergence of Shallow Neural Network Training with Randomly Masked Neuronscited by 11
  25. 2022Federated Select: A Primitive for Communication- and Memory-Efficient Federated Learningcited by 11
  26. 2021GIST: Distributed Training for Large-Scale Graph Convolutional Networkscited by 10
  27. 2021Efficient and Less Centralized Federated Learningcited by 8
  28. 2022LOFT: Finding Lottery Tickets through Filter-wise Trainingcited by 3

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 2.

4 papers.

  1. 2024Federated Unlearning: A Survey on Methods, Design Guidelines, and Evaluation Metricscited by 25
  2. 2024Effective Heterogeneous Federated Learning via Efficient Hypernetwork-based Weight Generationcited by 9
  3. 2025TwIST: Rigging the Lottery in Transformers with Independent Subnetwork Trainingcited by 1
  4. 2025Leveraging Label Potential for Enhanced Multimodal Emotion Recognitioncited by 0