FLea: Addressing Data Scarcity and Label Skew in Federated Learning via Privacy-preserving Feature Augmentation
Not on the map. Why?
Abstract (arXiv)
Federated Learning (FL) enables model development by leveraging data distributed across numerous edge devices without transferring local data to a central server. However, existing FL methods still face challenges when dealing with scarce and label-skewed data across devices, resulting in local model overfitting and drift, consequently hindering the performance of the global model. In response to these challenges, we propose a pioneering framework called textitFLea, incorporating the following key components: textiti) A global feature buffer that stores activation-target pairs shared from multiple clients to support local training. This design mitigates local model drift caused by the absence of certain classes; textitii) A feature augmentation approach based on local and global activation mix-ups for local training. This strategy enlarges the training samples, thereby reducing the risk of local overfitting; textitiii) An obfuscation method to minimize the correlation between intermediate activations and the source data, enhancing the privacy of shared features. To verify the superiority of textitFLea, we conduct extensive experiments using a wide range of data modalities, simulating different levels of local data scarcity and label skew. The results demonstrate that textitFLea consistently outperforms state-of-the-art FL counterparts (among 13 of the experimented 18 settings, the improvement is over 5%) while concurrently mitigating the privacy vulnerabilities associated with shared features. Code is available at https://github.com/XTxiatong/FLea.git
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 21 of its 39 references that are in the map. First posted on arXiv 4 Dec 2023.
Builds on
21 of 39 references.
References · 21
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
- 2018Federated Optimization in Heterogeneous Networkscited by 2,971
- 2016Deep Learning with Differential Privacycited by 2,682
- 2018Federated Learning with Non-IID Datacited by 1,099
- 2021Model-Contrastive Federated Learningcited by 762
- 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
- 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
- 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 349
- 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
9 more
- 2015Inverting Visual Representations with Convolutional Networkscited by 168
- 2021FedMix: Approximation of Mixup under Mean Augmented Federated Learningcited by 108
- 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 108
- 2021On Large-Cohort Training for Federated Learningcited by 73
- 2020NoPeek: Information leakage reduction to share activations in distributed deep learningcited by 66
- 2022Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learningcited by 52
- 2022FedBR: Improving Federated Learning on Heterogeneous Data via Local Learning Bias Reductioncited by 35
- 2023Confidence-aware Personalized Federated Learning via Variational Expectation Maximizationcited by 17
- 2016Supervised Dimensionality Reduction via Distance Correlation Maximizationcited by 7
Built on by
Citing papers per year: 2024: 1; 2025: 1; 2026: 1.
3 papers.