mlsanityData to 14 Oct 2026

Federated Multi-Objective Learning

arXiv 2310.09866 ↗First posted 15 Oct 2023cs.LG cs.AI cs.DCcited by 12

Not on the map. Why?

Abstract (arXiv)

In recent years, multi-objective optimization (MOO) emerges as a foundational problem underpinning many multi-agent multi-task learning applications. However, existing algorithms in MOO literature remain limited to centralized learning settings, which do not satisfy the distributed nature and data privacy needs of such multi-agent multi-task learning applications. This motivates us to propose a new federated multi-objective learning (FMOL) framework with multiple clients distributively and collaboratively solving an MOO problem while keeping their training data private. Notably, our FMOL framework allows a different set of objective functions across different clients to support a wide range of applications, which advances and generalizes the MOO formulation to the federated learning paradigm for the first time. For this FMOL framework, we propose two new federated multi-objective optimization (FMOO) algorithms called federated multi-gradient descent averaging (FMGDA) and federated stochastic multi-gradient descent averaging (FSMGDA). Both algorithms allow local updates to significantly reduce communication costs, while achieving the em same convergence rates as those of their algorithmic counterparts in the single-objective federated learning. Our extensive experiments also corroborate the efficacy of our proposed FMOO algorithms.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 33 of its 58 references that are in the map. First posted on arXiv 15 Oct 2023.

Builds on

33 of 58 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  4. 2019Advances and Open Problems in Federated Learningcited by 3,046
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  6. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,150
  7. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  8. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
  9. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 724
  10. 2017Dynamic Routing Between Capsulescited by 689
  11. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  12. 2021A Field Guide to Federated Optimizationcited by 315
21 more
  1. 2019Pareto Multi-Task Learningcited by 224
  2. 2018Don't Use Large Mini-Batches, Use Local SGDcited by 202
  3. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 193
  4. 2021Federated Multi-armed Bandits with Personalizationcited by 62
  5. 2022FedNest: Federated Bilevel, Minimax, and Compositional Optimizationcited by 60
  6. 2020Federated Learning Meets Multi-objective Optimizationcited by 53
  7. 2021Addressing Algorithmic Disparity and Performance Inconsistency in Federated Learningcited by 50
  8. 2019The stochastic multi-gradient algorithm for multi-objective optimization and its application to supervised machine learningcited by 46
  9. 2022Uncertainty-Aware Search Framework for Multi-Objective Bayesian Optimizationcited by 44
  10. 2021Anarchic Federated Learningcited by 42
  11. 2022Federated Minimax Optimization: Improved Convergence Analyses and Algorithmscited by 37
  12. 2023Direction-oriented Multi-objective Learning: Simple and Provable Stochastic Algorithmscited by 34
  13. 2022Federated Stochastic Approximation under Markov Noise and Heterogeneity: Applications in Reinforcement Learningcited by 27
  14. 2022Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Stochastic Approachcited by 14
  15. 2022NET-FLEET: Achieving Linear Convergence Speedup for Fully Decentralized Federated Learning with Heterogeneous Datacited by 12
  16. 2016Gradient-Based Multiobjective Optimization with Uncertaintiescited by 11
  17. 2022Multi-Label Learning to Rank through Multi-Objective Optimizationcited by 10
  18. 2022Taming Fat-Tailed ("Heavier-Tailed'' with Potentially Infinite Variance) Noise in Federated Learningcited by 10
  19. 2022Towards Multi-Objective Statistically Fair Federated Learningcited by 8
  20. 2020Multi-objective Ranking via Constrained Optimizationcited by 5
  21. 2020A Decentralized Multi-Objective Optimization Algorithmcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 5; 2026: 4.

12 papers.

  1. 2024MGDA Converges under Generalized Smoothness, Provablycited by 7
  2. 2024Federated Communication-Efficient Multi-Objective Optimizationcited by 3
  3. 2024FedDTPT: Federated Discrete and Transferable Prompt Tuning for Black-Box Large Language Modelscited by 1
  4. 2025Federated Self-Supervised Learning for One-Shot Cross-Modal and Cross-Imaging Technique Segmentationcited by 1
  5. 2025FIRM: Federated In-client Regularized Multi-objective Alignment for Large Language Modelscited by 1
  6. 2025Explainability and Continual Learning meet Federated Learning at the Network Edgecited by 0
  7. 2025Enabling Pareto-Stationarity Exploration in Multi-Objective Reinforcement Learning: A Multi-Objective Weighted-Chebyshev Actor-Critic Approachcited by 0
  8. 2025Federated Multi-Objective Learning with Controlled Pareto Frontierscited by 0
  9. 2026When the Server Steps In: Calibrated Updates for Fair Federated Learningcited by 0
  10. 2026FedCARE: A Multi-Objective Personalised Federated Learning Framework for Smart Healthcarecited by 0
  11. 2026A Momentum-Based Variance-Reduced Algorithm for Federated Multiobjective Optimizationcited by 0
  12. 2026Collaborative Personalized Preference Alignment for LLMs under Data Deficiencycited by 0