SCALA: Split Federated Learning with Concatenated Activations and Logit Adjustments
Not on the map. Why?
Abstract (arXiv)
Split Federated Learning (SFL) is a distributed machine learning framework which strategically divides the learning process between a server and clients and collaboratively trains a shared model by aggregating local models updated based on data from distributed clients. However, data heterogeneity and partial client participation result in label distribution skew, which severely degrades the learning performance. To address this issue, we propose SFL with Concatenated Activations and Logit Adjustments (SCALA). Specifically, the activations from the client-side models are concatenated as the input of the server-side model so as to centrally adjust label distribution across different clients, and logit adjustments of loss functions on both server-side and client-side models are performed to deal with the label distribution variation across different subsets of participating clients. Theoretical analysis and experimental results verify the superiority of the proposed SCALA on public datasets.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 61 of its 99 references that are in the map. First posted on arXiv 8 May 2024.
Builds on
61 of 99 references.
References · 61
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
- 2019Advances and Open Problems in Federated Learningcited by 3,021
- 2018Federated Optimization in Heterogeneous Networkscited by 2,971
- 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
- 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
- 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
- 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 806
- 2020Long-tail learning via logit adjustmentcited by 605
- 2021Federated Learning Based on Dynamic Regularizationcited by 555
38 more
- 2021Deep Long-Tailed Learning: A Surveycited by 416
- 2018Split learning for health: Distributed deep learning without sharing raw patient datacited by 391
- 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 349
- 2021Federated Learning on Non-IID Data: A Surveycited by 345
- 2020SplitFed: When Federated Learning Meets Split Learningcited by 308
- 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 303
- 2018Distributed learning of deep neural network over multiple agentscited by 277
- 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
- 2018Federated Learning via Over-the-Air Computationcited by 215
- 2021Improving Calibration for Long-Tailed Recognitioncited by 212
- 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 192
- 2020Minibatch vs Local SGD for Heterogeneous Distributed Learningcited by 147
- 2020Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition from a Domain Adaptation Perspectivecited by 143
- 2019Federated Learning for Wireless Communications: Motivation, Opportunities and Challengescited by 128
- 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 108
- 2018Towards an Intelligent Edge: Wireless Communication Meets Machine Learningcited by 94
- 2021Asynchronous Federated Learning on Heterogeneous Devices: A Surveycited by 93
- 2021Preservation of the Global Knowledge by Not-True Distillation in Federated Learningcited by 87
- 2020Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning Approachcited by 80
- 2021FedProc: Prototypical Contrastive Federated Learning on Non-IID datacited by 80
- 2019Detailed comparison of communication efficiency of split learning and federated learningcited by 76
- 2020NoPeek: Information leakage reduction to share activations in distributed deep learningcited by 66
- 2022Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Featurescited by 63
- 2020Scheduling for Cellular Federated Edge Learning with Importance and Channel Awarenesscited by 62
- 2022Client Selection in Federated Learning: Principles, Challenges, and Opportunitiescited by 62
- 2020Fast-Convergent Federated Learningcited by 61
- 2020Federated learning with class imbalance reductioncited by 52
- 2022FedFA: Federated Learning with Feature Anchors to Align Features and Classifiers for Heterogeneous Datacited by 49
- 2022A Unified Analysis of Federated Learning with Arbitrary Client Participationcited by 45
- 2022Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learningcited by 45
- 2022ResSFL: A Resistance Transfer Framework for Defending Model Inversion Attack in Split Federated Learningcited by 36
- 2023Optimal Resource Allocation for U-Shaped Parallel Split Learningcited by 33
- 2024An Aggregation-Free Federated Learning for Tackling Data Heterogeneitycited by 33
- 2022Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reductioncited by 32
- 2023On the Convergence of Federated Averaging with Cyclic Client Participationcited by 31
- 2021Cost-Effective Federated Learning in Mobile Edge Networkscited by 25
- 2023The Best of Both Worlds: Accurate Global and Personalized Models through Federated Learning with Data-Free Hyper-Knowledge Distillationcited by 25
- 2022Communication-Efficient Federated Learning with Accelerated Client Gradientcited by 24
50 most cited of 61 All 61 (JSON)
Built on by
Citing papers per year: 2024: 1; 2025: 1.
2 papers.