mlsanityData to 7 Oct 2026

Exploiting Label Skews in Federated Learning with Model Concatenation

arXiv 2312.06290 ↗First posted 11 Dec 2023cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) has emerged as a promising solution to perform deep learning on different data owners without exchanging raw data. However, non-IID data has been a key challenge in FL, which could significantly degrade the accuracy of the final model. Among different non-IID types, label skews have been challenging and common in image classification and other tasks. Instead of averaging the local models in most previous studies, we propose FedConcat, a simple and effective approach that concatenates these local models as the base of the global model to effectively aggregate the local knowledge. To reduce the size of the global model, we adopt the clustering technique to group the clients by their label distributions and collaboratively train a model inside each cluster. We theoretically analyze the advantage of concatenation over averaging by analyzing the information bottleneck of deep neural networks. Experimental results demonstrate that FedConcat achieves significantly higher accuracy than previous state-of-the-art FL methods in various heterogeneous label skew distribution settings and meanwhile has lower communication costs. Our code is publicly available at https://github.com/sjtudyq/FedConcat.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 33 of its 51 references that are in the map. First posted on arXiv 11 Dec 2023.

Builds on

33 of 51 references.

References · 33

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  5. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  6. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  7. 2019Towards Federated Learning at Scale: System Designcited by 826
  8. 2015Deep Learning and the Information Bottleneck Principlecited by 793
  9. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  10. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
  11. 2020Federated Learning with Matched Averagingcited by 564
  12. 2020An Efficient Framework for Clustered Federated Learningcited by 505
21 more
  1. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  2. 2017Opening the Black Box of Deep Neural Networks via Informationcited by 494
  3. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 371
  4. 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 222
  5. 2019SCAFFOLD: Stochastic Controlled Averaging for Federated Learningcited by 174
  6. 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
  7. 2021Local Learning Matters: Rethinking Data Heterogeneity in Federated Learningcited by 114
  8. 2021Multi-Center Federated Learning: Clients Clustering for Better Personalizationcited by 112
  9. 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 108
  10. 2020Robust Federated Learning: The Case of Affine Distribution Shiftscited by 102
  11. 2020FedGAN: Federated Generative Adversarial Networks for Distributed Datacited by 89
  12. 2020Privacy Preserving Vertical Federated Learning for Tree-based Modelscited by 89
  13. 2020Federated Bayesian Optimization via Thompson Samplingcited by 77
  14. 2019Active Federated Learningcited by 63
  15. 2020Federated Learning with Only Positive Labelscited by 55
  16. 2019Practical Federated Gradient Boosting Decision Treescited by 53
  17. 2021Flexible Clustered Federated Learning for Client-Level Data Distribution Shiftcited by 47
  18. 2021FedSoft: Soft Clustered Federated Learning with Proximal Local Updatingcited by 46
  19. 2021FedCor: Correlation-Based Active Client Selection Strategy for Heterogeneous Federated Learningcited by 43
  20. 2020FedCluster: Boosting the Convergence of Federated Learning via Cluster-Cyclingcited by 26
  21. 2020Federated Learning of User Authentication Modelscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 2; 2026: 5.

14 papers.

Sort:

  1. 2024Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directionscited by 12
  2. 2024Exploring Vacant Classes in Label-Skewed Federated Learningcited by 10
  3. 2024Optimizing Federated Learning by Entropy-Based Client Selectioncited by 3
  4. 2024SCALA: Split Federated Learning with Concatenated Activations and Logit Adjustmentscited by 2
  5. 2024Exploiting Label Skewness for Spiking Neural Networks in Federated Learningcited by 2
  6. 2024Rethinking the Starting Point: Collaborative Pre-Training for Federated Downstream Taskscited by 1
  7. 2025High-Energy Concentration for Federated Learning in Frequency Domaincited by 1
  8. 2025Distribution-Controlled Client Selection to Improve Federated Learning Strategiescited by 1
  9. 2026Single-Round Clustered Federated Learning via Data Collaboration Analysis for Non-IID Datacited by 1
  10. 2024The Key of Parameter Skew in Federated Learningcited by 0
  11. 2026Breaking the Prototype Bias Loop: Confidence-Aware Federated Contrastive Learning for Highly Imbalanced Clientscited by 0
  12. 2026From Data Heterogeneity to Convergence: A Data-Centric Review of Federated Learningcited by 0
2 more
  1. 2026WHERE to Generate Matters: Budget-Aware Synthetic Augmentation for Label Skewed Federated Learningcited by 0
  2. 2026FedCC: Towards Addressing Label Distribution Skews in Distillation-Based Federated Learningcited by 0