mlsanityData to 14 Oct 2026

FLea: Addressing Data Scarcity and Label Skew in Federated Learning via Privacy-preserving Feature Augmentation

arXiv 2406.09547 ↗First posted 13 Jun 2024cs.LG cs.DCcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) enables model development by leveraging data distributed across numerous edge devices without transferring local data to a central server. However, existing FL methods still face challenges when dealing with scarce and label-skewed data across devices, resulting in local model overfitting and drift, consequently hindering the performance of the global model. In response to these challenges, we propose a pioneering framework called FLea, incorporating the following key components: i) A global feature buffer that stores activation-target pairs shared from multiple clients to support local training. This design mitigates local model drift caused by the absence of certain classes; ii) A feature augmentation approach based on local and global activation mix-ups for local training. This strategy enlarges the training samples, thereby reducing the risk of local overfitting; iii) An obfuscation method to minimize the correlation between intermediate activations and the source data, enhancing the privacy of shared features. To verify the superiority of FLea, we conduct extensive experiments using a wide range of data modalities, simulating different levels of local data scarcity and label skew. The results demonstrate that FLea consistently outperforms state-of-the-art FL counterparts (among 13 of the experimented 18 settings, the improvement is over 5% while concurrently mitigating the privacy vulnerabilities associated with shared features. Code is available at https://github.com/XTxiatong/FLea.git.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 39 references that are in the map. First posted on arXiv 13 Jun 2024.

Builds on

21 of 39 references.

References · 21

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  4. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  6. 2016Deep Learning with Differential Privacycited by 2,728
  7. 2018Federated Learning with Non-IID Datacited by 1,108
  8. 2021Model-Contrastive Federated Learningcited by 766
  9. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
  10. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 387
  11. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 352
  12. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
9 more
  1. 2015Inverting Visual Representations with Convolutional Networkscited by 170
  2. 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 109
  3. 2021FedMix: Approximation of Mixup under Mean Augmented Federated Learningcited by 108
  4. 2021On Large-Cohort Training for Federated Learningcited by 74
  5. 2020NoPeek: Information leakage reduction to share activations in distributed deep learningcited by 66
  6. 2022Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learningcited by 52
  7. 2022FedBR: Improving Federated Learning on Heterogeneous Data via Local Learning Bias Reductioncited by 35
  8. 2023Confidence-aware Personalized Federated Learning via Variational Expectation Maximizationcited by 17
  9. 2016Supervised Dimensionality Reduction via Distance Correlation Maximizationcited by 7

Built on by

Not cited yet.