mlsanityData to 7 Oct 2026

Dynamic Attention-based Communication-Efficient Federated Learning

arXiv 2108.05765 ↗First posted 12 Aug 2021cs.LG cs.DCcited by 9

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) offers a solution to train a global machine learning model while still maintaining data privacy, without needing access to data stored locally at the clients. However, FL suffers performance degradation when client data distribution is non-IID, and a longer training duration to combat this degradation may not necessarily be feasible due to communication limitations. To address this challenge, we propose a new adaptive training algorithm textttAdaFL, which comprises two components: (i) an attention-based client selection mechanism for a fairer training scheme among the clients; and (ii) a dynamic fraction method to balance the trade-off between performance stability and communication efficiency. Experimental results show that our textttAdaFL algorithm outperforms the usual textttFedAvg algorithm, and can be incorporated to further improve various state-of-the-art FL algorithms, with respect to three aspects: model accuracy, performance stability, and communication efficiency.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 20 of its 25 references that are in the map. First posted on arXiv 12 Aug 2021.

Builds on

20 of 25 references.

References · 20

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  5. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  6. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  7. 2018Federated Learning with Non-IID Datacited by 1,099
  8. 2017Federated Multi-Task Learningcited by 669
  9. 2020Federated Learning with Matched Averagingcited by 564
  10. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
  11. 2019Agnostic Federated Learningcited by 451
  12. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
8 more
  1. 2019Fair Resource Allocation in Federated Learningcited by 389
  2. 2020Adaptive Personalized Federated Learningcited by 353
  3. 2019Asynchronous Federated Optimizationcited by 264
  4. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 242
  5. 2021FedMix: Approximation of Mixup under Mean Augmented Federated Learningcited by 108
  6. 2019Robust Federated Learning in a Heterogeneous Environmentcited by 88
  7. 2018Learning Private Neural Language Modeling with Attentive Aggregationcited by 49
  8. 2019Incentive Design for Efficient Federated Learning in Mobile Networks: A Contract Theory Approachcited by 45

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 3; 2024: 2; 2025: 2.

9 papers.

  1. 2022FedCorr: Multi-Stage Federated Learning for Label Noise Correctioncited by 61
  2. 2023Fed-GraB: Federated Long-tailed Learning with Self-Adjusting Gradient Balancercited by 17
  3. 2023Towards Interpretable Federated Learningcited by 11
  4. 2024Spectral Co-Distillation for Personalized Federated Learningcited by 8
  5. 2024Exploring the Practicality of Federated Learning: A Survey Towards the Communication Perspectivecited by 4
  6. 2022Online Meta-Learning for Model Update Aggregation in Federated Learning for Click-Through Rate Predictioncited by 2
  7. 2023When to Trust Aggregated Gradients: Addressing Negative Client Sampling in Federated Learningcited by 0
  8. 2025Communication-Efficient Federated Learning with Adaptive Number of Participantscited by 0
  9. 2025Privacy Preserved Federated Learning with Attention-Based Aggregation for Biometric Recognitioncited by 0