# Federated Multi-Objective Learning

arXiv: https://arxiv.org/abs/2310.09866 · HTML: https://mlsanity.com/paper/2310.09866/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 58 references that are in the map.
- First posted on arXiv 15 Oct 2023.

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [Multi-Task Learning as Multi-Objective Optimization](https://mlsanity.com/paper/1810.04650/) (2018)
- [Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming](https://mlsanity.com/paper/1309.5549/) (2013)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [Federated Learning Based on Dynamic Regularization](https://mlsanity.com/paper/2111.04263/) (2021)
- [A Field Guide to Federated Optimization](https://mlsanity.com/paper/2107.06917/) (2021)
- [Pareto Multi-Task Learning](https://mlsanity.com/paper/1912.12854/) (2019)
- [Don't Use Large Mini-Batches, Use Local SGD](https://mlsanity.com/paper/1808.07217/) (2018)
- [Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learning](https://mlsanity.com/paper/2101.11203/) (2021)
- [Federated Multi-armed Bandits with Personalization](https://mlsanity.com/paper/2102.13101/) (2021)
- [FedNest: Federated Bilevel, Minimax, and Compositional Optimization](https://mlsanity.com/paper/2205.02215/) (2022)
- [Federated Learning Meets Multi-objective Optimization](https://mlsanity.com/paper/2006.11489/) (2020)
- [Addressing Algorithmic Disparity and Performance Inconsistency in Federated Learning](https://mlsanity.com/paper/2108.08435/) (2021)
- [The stochastic multi-gradient algorithm for multi-objective optimization and its application to supervised machine learning](https://mlsanity.com/paper/1907.04472/) (2019)
- [Uncertainty-Aware Search Framework for Multi-Objective Bayesian Optimization](https://mlsanity.com/paper/2204.05944/) (2022)
- [Anarchic Federated Learning](https://mlsanity.com/paper/2108.09875/) (2021)
- [Federated Minimax Optimization: Improved Convergence Analyses and Algorithms](https://mlsanity.com/paper/2203.04850/) (2022)
- [Direction-oriented Multi-objective Learning: Simple and Provable Stochastic Algorithms](https://mlsanity.com/paper/2305.18409/) (2023)
- [Federated Stochastic Approximation under Markov Noise and Heterogeneity: Applications in Reinforcement Learning](https://mlsanity.com/paper/2206.10185/) (2022)
- [Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Stochastic Approach](https://mlsanity.com/paper/2210.12624/) (2022)
- [NET-FLEET: Achieving Linear Convergence Speedup for Fully Decentralized Federated Learning with Heterogeneous Data](https://mlsanity.com/paper/2208.08490/) (2022)
- [Gradient-Based Multiobjective Optimization with Uncertainties](https://mlsanity.com/paper/1612.03815/) (2016)
- [Multi-Label Learning to Rank through Multi-Objective Optimization](https://mlsanity.com/paper/2207.03060/) (2022)
- [Taming Fat-Tailed ("Heavier-Tailed'' with Potentially Infinite Variance) Noise in Federated Learning](https://mlsanity.com/paper/2210.00690/) (2022)

## Built on by (most cited first, 12 of 12)

- [MGDA Converges under Generalized Smoothness, Provably](https://mlsanity.com/paper/2405.19440/) (2024)
- [Federated Communication-Efficient Multi-Objective Optimization](https://mlsanity.com/paper/2410.16398/) (2024)
- [FedDTPT: Federated Discrete and Transferable Prompt Tuning for Black-Box Large Language Models](https://mlsanity.com/paper/2411.00985/) (2024)
- [Federated Self-Supervised Learning for One-Shot Cross-Modal and Cross-Imaging Technique Segmentation](https://mlsanity.com/paper/2503.23507/) (2025)
- [FIRM: Federated In-client Regularized Multi-objective Alignment for Large Language Models](https://mlsanity.com/paper/2511.16992/) (2025)
- [Explainability and Continual Learning meet Federated Learning at the Network Edge](https://mlsanity.com/paper/2504.08536/) (2025)
- [Enabling Pareto-Stationarity Exploration in Multi-Objective Reinforcement Learning: A Multi-Objective Weighted-Chebyshev Actor-Critic Approach](https://mlsanity.com/paper/2507.21397/) (2025)
- [Federated Multi-Objective Learning with Controlled Pareto Frontiers](https://mlsanity.com/paper/2508.05424/) (2025)
- [When the Server Steps In: Calibrated Updates for Fair Federated Learning](https://mlsanity.com/paper/2601.05352/) (2026)
- [FedCARE: A Multi-Objective Personalised Federated Learning Framework for Smart Healthcare](https://mlsanity.com/paper/2608.03498/) (2026)
- [A Momentum-Based Variance-Reduced Algorithm for Federated Multiobjective Optimization](https://mlsanity.com/paper/2608.22945/) (2026)
- [Collaborative Personalized Preference Alignment for LLMs under Data Deficiency](https://mlsanity.com/paper/2610.05898/) (2026)

## Abstract (arXiv)

In recent years, multi-objective optimization (MOO) emerges as a foundational problem underpinning many multi-agent multi-task learning applications. However, existing algorithms in MOO literature remain limited to centralized learning settings, which do not satisfy the distributed nature and data privacy needs of such multi-agent multi-task learning applications. This motivates us to propose a new federated multi-objective learning (FMOL) framework with multiple clients distributively and collaboratively solving an MOO problem while keeping their training data private. Notably, our FMOL framework allows a different set of objective functions across different clients to support a wide range of applications, which advances and generalizes the MOO formulation to the federated learning paradigm for the first time. For this FMOL framework, we propose two new federated multi-objective optimization (FMOO) algorithms called federated multi-gradient descent averaging (FMGDA) and federated stochastic multi-gradient descent averaging (FSMGDA). Both algorithms allow local updates to significantly reduce communication costs, while achieving the em same convergence rates as those of their algorithmic counterparts in the single-objective federated learning. Our extensive experiments also corroborate the efficacy of our proposed FMOO algorithms.