Feature-based Federated Transfer Learning: Communication Efficiency, Robustness and Privacy
Not on the map. Why?
Abstract (arXiv)
In this paper, we propose feature-based federated transfer learning as a novel approach to improve communication efficiency by reducing the uplink payload by multiple orders of magnitude compared to that of existing approaches in federated learning and federated transfer learning. Specifically, in the proposed feature-based federated learning, we design the extracted features and outputs to be uploaded instead of parameter updates. For this distributed learning model, we determine the required payload and provide comparisons with the existing schemes. Subsequently, we analyze the robustness of feature-based federated transfer learning against packet loss, data insufficiency, and quantization. Finally, we address privacy considerations by defining and analyzing label privacy leakage and feature privacy leakage, and investigating mitigating approaches. For all aforementioned analyses, we evaluate the performance of the proposed learning scheme via experiments on an image classification task and a natural language processing task to demonstrate its effectiveness.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 54 of its 96 references that are in the map. First posted on arXiv 15 May 2024.
Builds on
54 of 96 references.
References · 54
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
- 2019Advances and Open Problems in Federated Learningcited by 3,021
- 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
- 2016Deep Learning with Differential Privacycited by 2,682
- 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
- 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
- 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
- 2018Federated Learning with Non-IID Datacited by 1,099
- 2019Deep Leakage from Gradientscited by 1,004
38 more
- 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 707
- 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
- 2020Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentationcited by 595
- 2019Segmentation Transformer: Object-Contextual Representations for Semantic Segmentationcited by 573
- 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 566
- 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 561
- 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 558
- 2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 530
- 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 528
- 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
- 2018Split learning for health: Distributed deep learning without sharing raw patient datacited by 391
- 2018Sparsified SGD with Memorycited by 373
- 2020iDLG: Improved Deep Leakage from Gradientscited by 367
- 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
- 2020LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attentioncited by 326
- 2020Personalized Federated Learning: A Meta-Learning Approachcited by 309
- 2020SplitFed: When Federated Learning Meets Split Learningcited by 308
- 2018Distributed learning of deep neural network over multiple agentscited by 277
- 2018The Convergence of Sparsified Gradient Methodscited by 228
- 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 219
- 2018Federated Learning via Over-the-Air Computationcited by 215
- 2021Evaluating Gradient Inversion Attacks and Defenses in Federated Learningcited by 196
- 2020Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognitioncited by 185
- 2019FedHealth: A Federated Transfer Learning Framework for Wearable Healthcarecited by 182
- 2019Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computationscited by 179
- 2021EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedbackcited by 131
- 2021Distributed Learning in Wireless Networks: Recent Progress and Future Challengescited by 125
- 2021Gradient Inversion with Generative Image Priorcited by 101
- 2018Wireless Network Intelligence at the Edgecited by 90
- 2021SeMask: Semantically Masked Transformers for Semantic Segmentationcited by 55
- 2021MSRF-Net: A Multi-Scale Residual Fusion Network for Biomedical Image Segmentationcited by 53
- 2021Communication Efficiency in Federated Learning: Achievements and Challengescited by 40
- 2019Wireless Federated Distillation for Distributed Edge Learning with Heterogeneous Datacited by 39
- 2020Federated Transfer Learning for EEG Signal Classificationcited by 36
- 2020Pruning Algorithms to Accelerate Convolutional Neural Networks for Edge Applications: A Surveycited by 26
- 2021Time-Correlated Sparsification for Communication-Efficient Federated Learningcited by 25
- 2021User-Level Label Leakage from Gradients in Federated Learningcited by 21
- 2021Partial Variable Training for Efficient On-Device Federated Learningcited by 14
50 most cited of 54 All 54 (JSON)
Built on by
Citing papers per year: 2025: 1.
1 paper.