mlsanityData to 7 Oct 2026

FedProphet: Memory-Efficient Federated Adversarial Training via Robust and Consistent Cascade Learning

arXiv 2409.08372 ↗First posted 12 Sep 2024cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated Adversarial Training (FAT) can supplement robustness against adversarial examples to Federated Learning (FL), promoting a meaningful step toward trustworthy AI. However, FAT requires large models to preserve high accuracy while achieving strong robustness, incurring high memory-swapping latency when training on memory-constrained edge devices. Existing memory-efficient FL methods suffer from poor accuracy and weak robustness due to inconsistent local and global models. In this paper, we propose FedProphet, a novel FAT framework that can achieve memory efficiency, robustness, and consistency simultaneously. FedProphget reduces the memory requirement in local training while guaranteeing adversarial robustness by adversarial cascade learning with strong convexity regularization, and we show that the strong robustness also implies low inconsistency in FedProphet. We also develop a training coordinator on the server of FL, with Adaptive Perturbation Adjustment for utility-robustness balance and Differentiated Module Assignment for objective inconsistency mitigation. FedPeophet significantly outperforms other baselines under different experimental settings, maintaining the accuracy and robustness of end-to-end FAT with 80% memory reduction and up to 10.8x speedup in training time.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 35 of its 44 references that are in the map. First posted on arXiv 12 Sep 2024.

Builds on

35 of 44 references.

References · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  6. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  7. 2019Advances and Open Problems in Federated Learningcited by 3,021
  8. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  9. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  10. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  11. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  12. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
23 more
  1. 2020Fast is better than free: Revisiting adversarial trainingcited by 723
  2. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  3. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  4. 2019FreeLB: Enhanced Adversarial Training for Natural Language Understandingcited by 261
  5. 2015Federated Optimization:Distributed Optimization Beyond the Datacentercited by 193
  6. 2018Robustness via curvature regularization, and vice versacited by 174
  7. 2019SCAFFOLD: Stochastic Controlled Averaging for Federated Learningcited by 174
  8. 2018Expanding the Reach of Federated Learning by Reducing Client Resource Requirementscited by 162
  9. 2021On the Convergence and Robustness of Adversarial Trainingcited by 127
  10. 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
  11. 2022Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learningcited by 81
  12. 2019Decoupled Greedy Learning of CNNscited by 76
  13. 2019Interpreting Adversarially Trained Convolutional Neural Networkscited by 75
  14. 2020DVERGE: Diversifying Vulnerabilities for Enhanced Robust Generation of Ensemblescited by 73
  15. 2021Revisiting Locally Supervised Learning: an Alternative to End-to-end Trainingcited by 71
  16. 2021Federated Dropout -- A Simple Approach for Enabling Federated Learning on Resource Constrained Devicescited by 56
  17. 2023ZeRO++: Extremely Efficient Collective Communication for Giant Model Trainingcited by 50
  18. 2021FedCor: Correlation-Based Active Client Selection Strategy for Heterogeneous Federated Learningcited by 43
  19. 2022Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reductioncited by 32
  20. 2021Adversarial training in communication constrained federated learningcited by 17
  21. 2017Forward Thinking: Building and Training Neural Networks One Layer at a Timecited by 15
  22. 2021Federated Robustness Propagation: Sharing Robustness in Heterogeneous Federated Learningcited by 9
  23. 2020Exploring the Potential of Low-bit Training of Convolutional Neural Networkscited by 7

Built on by

Not cited yet.