mlsanityData to 7 Oct 2026

Federated Deep AUC Maximization for Heterogeneous Data with a Constant Communication Complexity

arXiv 2102.04635 ↗First posted 9 Feb 2021cs.LG cs.DC math.OCcited by 21

Not on the map. Why?

Abstract (arXiv)

Deep AUC (area under the ROC curve) Maximization (DAM) has attracted much attention recently due to its great potential for imbalanced data classification. However, the research on Federated Deep AUC Maximization (FDAM) is still limited. Compared with standard federated learning (FL) approaches that focus on decomposable minimization objectives, FDAM is more complicated due to its minimization objective is non-decomposable over individual examples. In this paper, we propose improved FDAM algorithms for heterogeneous data by solving the popular non-convex strongly-concave min-max formulation of DAM in a distributed fashion, which can also be applied to a class of non-convex strongly-concave min-max problems. A striking result of this paper is that the communication complexity of the proposed algorithm is a constant independent of the number of machines and also independent of the accuracy level, which improves an existing result by orders of magnitude. The experiments have demonstrated the effectiveness of our FDAM algorithm on benchmark datasets, and on medical chest X-ray images from different organizations. Our experiment shows that the performance of FDAM using data from multiple hospitals can improve the AUC score on testing data from a single hospital for detecting life-threatening diseases based on chest radiographs. The proposed method is implemented in our open-sourced library LibAUC (www.libauc.org) whose github address is https://github.com/Optimization-AI/ICML2021_FedDeepAUC_CODASCA.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 29 of its 49 references that are in the map. First posted on arXiv 9 Feb 2021.

Builds on

29 of 49 references.

References · 29

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  4. 2017A Survey on Deep Learning in Medical Image Analysiscited by 1,182
  5. 2020The Future of Digital Health with Federated Learningcited by 625
  6. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  7. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
  8. 2019PadChest: A large chest x-ray image dataset with multi-label annotated reportscited by 308
  9. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 277
  10. 2019On Gradient Descent Ascent for Nonconvex-Concave Minimax Problemscited by 266
  11. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  12. 2020VinDr-CXR: An open dataset of chest X-rays with radiologist's annotationscited by 236
17 more
  1. 2018Don't Use Large Mini-Batches, Use Local SGDcited by 201
  2. 2019On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimizationcited by 178
  3. 2019SCAFFOLD: Stochastic Controlled Averaging for Federated Learningcited by 174
  4. 2020Is Local SGD Better than Minibatch SGD?cited by 151
  5. 2020Minibatch vs Local SGD for Heterogeneous Distributed Learningcited by 147
  6. 2019Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronizationcited by 106
  7. 2016Deep Learning for Identifying Metastatic Breast Cancercited by 93
  8. 2020Stochastic Recursive Gradient Descent Ascent for Stochastic Nonconvex-Strongly-Concave Minimax Problemscited by 75
  9. 2019Stochastic AUC Maximization with Deep Neural Networkscited by 61
  10. 2015Communication Complexity of Distributed Convex Learning and Optimizationcited by 59
  11. 2019Can we trust deep learning models diagnosis? The impact of domain shift in chest radiograph classificationcited by 57
  12. 2018Efficient Decentralized Deep Learning by Dynamic Model Averagingcited by 30
  13. 2020Communication-Efficient Distributed Stochastic AUC Maximization with Deep Neural Networkscited by 27
  14. 2017Skin Lesion Analysis Towards Melanoma Detection Using Deep Learning Networkcited by 21
  15. 2020Fast Objective & Duality Gap Convergence for Non-Convex Strongly-Concave Min-Max Problems with PL Conditioncited by 19
  16. 2015Experiments on Parallel Training of Deep Neural Network using Model Averagingcited by 14
  17. 2014Parallel training of DNNs with Natural Gradient and Parameter Averagingcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 11; 2023: 4; 2024: 0; 2025: 1; 2026: 2.

21 papers.

Sort:

  1. 2022AUC Maximization in the Era of Big Data and AI: A Surveycited by 100
  2. 2021The Complexity of Nonconvex-Strongly-Concave Minimax Optimizationcited by 44
  3. 2022Federated Minimax Optimization: Improved Convergence Analyses and Algorithmscited by 37
  4. 2022When AUC meets DRO: Optimizing Partial AUC for Deep Learning with Non-Convex Convergence Guaranteecited by 29
  5. 2022Near-Optimal Algorithms for Making the Gradient Small in Stochastic Minimax Optimizationcited by 19
  6. 2022Optimizing Two-way Partial AUC with an End-to-end Frameworkcited by 15
  7. 2022Anchor Sampling for Federated Learning with Partial Client Participationcited by 14
  8. 2022FeDXL: Provable Federated Learning for Deep X-Risk Optimizationcited by 10
  9. 2023Solving a Class of Non-Convex Minimax Optimization in Federated Learningcited by 10
  10. 2023Federated Compositional Deep AUC Maximizationcited by 8
  11. 2021Deep AUC Maximization for Medical Image Classification: Challenges and Opportunitiescited by 6
  12. 2022An Efficient Stochastic Algorithm for Decentralized Nonconvex-Strongly-Concave Minimax Optimizationcited by 6
9 more
  1. 2022Adaptive Federated Minimax Optimization with Lower Complexitiescited by 5
  2. 2022Differentiated Federated Reinforcement Learning Based Traffic Offloading on Space-Air-Ground Integrated Networkscited by 4
  3. 2023Federated Conditional Stochastic Optimizationcited by 4
  4. 2023Serverless Federated AUPRC Optimization for Multi-Party Collaborative Imbalanced Data Miningcited by 3
  5. 2021CDMA: A Practical Cross-Device Federated Learning Algorithm for General Minimax Problemscited by 2
  6. 2025Stochastic Primal-Dual Double Block-Coordinate for Two-way Partial AUC Maximizationcited by 1
  7. 2022SAGDA: Achieving O(ε-2) Communication Complexity in Federated Min-Max Learningcited by 0
  8. 2026Communication-Efficient Federated AUC Maximization with Cyclic Client Participationcited by 0
  9. 2026Provably Communication-Efficient and Privacy-Preserving Federated Graph Neural Networkscited by 0