mlsanityData to 14 Oct 2026

Federated Communication-Efficient Multi-Objective Optimization

arXiv 2410.16398 ↗First posted 21 Oct 2024cs.LG cs.DCcited by 3

Not on the map. Why?

Abstract (arXiv)

We study a federated version of multi-objective optimization (MOO), where a single model is trained to optimize multiple objective functions. MOO has been extensively studied in the centralized setting but is less explored in federated or distributed settings. We propose FedCMOO, a novel communication-efficient federated multi-objective optimization (FMOO) algorithm that improves the error convergence performance of the model compared to existing approaches. Unlike prior works, the communication cost of FedCMOO does not scale with the number of objectives, as each client sends a single aggregated gradient to the central server. We provide a convergence analysis of the proposed method for smooth and non-convex objective functions under milder assumptions than in prior work. In addition, we introduce a variant of FedCMOO that allows users to specify a preference over the objectives in terms of a desired ratio of the final objective values. Through extensive experiments, we demonstrate the superiority of our proposed method over baseline approaches.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 59 of its 93 references that are in the map. First posted on arXiv 21 Oct 2024.

Builds on

59 of 93 references.

References · 59

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  4. 2019Advances and Open Problems in Federated Learningcited by 3,046
  5. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  6. 2017A Survey on Multi-Task Learningcited by 840
  7. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
  8. 2017Federated Multi-Task Learningcited by 672
  9. 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 564
  10. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  11. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 459
  12. 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 375
38 more
  1. 2020Multi-Task Learning with Deep Neural Networks: A Surveycited by 375
  2. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 356
  3. 2019Pareto Multi-Task Learningcited by 224
  4. 2021Efficiently Identifying Task Groupings for Multi-Task Learningcited by 222
  5. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 192
  6. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 174
  7. 2020A Survey on Negative Transfercited by 122
  8. 2022Multi-Objective GFlowNetscited by 101
  9. 2022AUC Maximization in the Era of Big Data and AI: A Surveycited by 100
  10. 2023FedExP: Speeding Up Federated Averaging via Extrapolationcited by 69
  11. 2022In Defense of the Unitary Scalarization for Deep Multi-Task Learningcited by 67
  12. 2022FedVARP: Tackling the Variance Due to Partial Client Participation in Federated Learningcited by 64
  13. 2022Do Current Multi-Task Optimization Methods in Deep Learning Even Help?cited by 60
  14. 2022PD-MORL: Preference-Driven Multi-Objective Reinforcement Learning Algorithmcited by 58
  15. 2018A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Taskscited by 54
  16. 2020Federated Learning Meets Multi-objective Optimizationcited by 53
  17. 2020Multi-Task Federated Learning for Personalised Deep Neural Networks in Edge Computingcited by 52
  18. 2022Pareto Set Learning for Expensive Multi-Objective Optimizationcited by 51
  19. 2023Revisiting Scalarization in Multi-Task Learning: A Theoretical Perspectivecited by 51
  20. 2021Addressing Algorithmic Disparity and Performance Inconsistency in Federated Learningcited by 50
  21. 2019The stochastic multi-gradient algorithm for multi-objective optimization and its application to supervised machine learningcited by 46
  22. 2021Scalable Pareto Front Approximation for Deep Multi-Objective Learningcited by 45
  23. 2022Uncertainty-Aware Search Framework for Multi-Objective Bayesian Optimizationcited by 44
  24. 2022Machine Learning Model Sizes and the Parameter Gapcited by 39
  25. 2023Three-Way Trade-Off in Multi-Objective Learning: Optimization, Generalization and Conflict-Avoidancecited by 36
  26. 2023Direction-oriented Multi-objective Learning: Simple and Provable Stochastic Algorithmscited by 34
  27. 2022Pareto Manifold Learning: Tackling multiple tasks via ensembles of single-task modelscited by 29
  28. 2022LibMTL: A Python Library for Multi-Task Learningcited by 25
  29. 2022A Survey on Federated Recommendation Systemscited by 25
  30. 2022Controllable Dynamic Multi-Task Architecturescited by 21
  31. 2019Multi-Gradient Descent for Multi-Objective Recommender Systemscited by 20
  32. 2021Federated Learning of Molecular Properties with Graph Neural Networks in a Heterogeneous Settingcited by 19
  33. 2023Scalarization for Multi-Task and Multi-Domain Learning at Scalecited by 19
  34. 2023FedHCA2: Towards Hetero-Client Federated Multi-Task Learningcited by 17
  35. 2024Few for Many: Tchebycheff Set Scalarization for Many-Objective Optimizationcited by 16
  36. 2022Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Stochastic Approachcited by 14
  37. 2023FedBone: Towards Large-Scale Federated Multi-Task Learningcited by 14
  38. 2023Federated Multi-Objective Learningcited by 12

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyondcited by 18
  2. 2025FIRM: Federated In-client Regularized Multi-objective Alignment for Large Language Modelscited by 1
  3. 2026A Momentum-Based Variance-Reduced Algorithm for Federated Multiobjective Optimizationcited by 0