# Fair Concurrent Training of Multiple Models in Federated Learning

arXiv: https://arxiv.org/abs/2404.13841 · HTML: https://mlsanity.com/paper/2404.13841/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 43 references that are in the map.
- First posted on arXiv 22 Apr 2024.

## Builds on (most cited first, 19 of 19)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [Fair Resource Allocation in Federated Learning](https://mlsanity.com/paper/1905.10497/) (2019)
- [Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge](https://mlsanity.com/paper/1804.08333/) (2018)
- [Federated Learning for Smart Healthcare: A Survey](https://mlsanity.com/paper/2111.08834/) (2021)
- [Multi-Armed Bandit Based Client Scheduling for Federated Learning](https://mlsanity.com/paper/2007.02315/) (2020)
- [FedSoft: Soft Clustered Federated Learning with Proximal Local Updating](https://mlsanity.com/paper/2112.06053/) (2021)
- [FedCross: Towards Accurate Federated Learning via Multi-Model Cross-Aggregation](https://mlsanity.com/paper/2210.08285/) (2022)
- [Multi-Job Intelligent Scheduling with Cross-Device Federated Learning](https://mlsanity.com/paper/2211.13430/) (2022)
- [MergeSFL: Split Federated Learning with Feature Merging and Batch Size Regulation](https://mlsanity.com/paper/2311.13348/) (2023)
- [Toward Multiple Federated Learning Services Resource Sharing in Mobile Edge Networks](https://mlsanity.com/paper/2011.12469/) (2020)
- [Adaptive Configuration for Heterogeneous Participants in Decentralized Federated Learning](https://mlsanity.com/paper/2212.02136/) (2022)
- [FedServing: A Federated Prediction Serving Framework Based on Incentive Mechanism](https://mlsanity.com/paper/2012.10566/) (2020)
- [Fairness-aware Crowdsourcing of IoT Energy Services](https://mlsanity.com/paper/2111.06064/) (2021)
- [Multi-Model Federated Learning with Provable Guarantees](https://mlsanity.com/paper/2207.04330/) (2022)
- [Multi-Model Federated Learning](https://mlsanity.com/paper/2201.02582/) (2022)
- [Fair Task Allocation in Crowdsourced Delivery](https://mlsanity.com/paper/1807.02987/) (2018)
- [Dynamic Pricing for Client Recruitment in Federated Learning](https://mlsanity.com/paper/2203.03192/) (2022)
- [FedRule: Federated Rule Recommendation System with Graph Neural Networks](https://mlsanity.com/paper/2211.06812/) (2022)

## Built on by (most cited first, 3 of 3)

- [FedBaF: Federated Learning Aggregation Biased by a Foundation Model](https://mlsanity.com/paper/2410.18352/) (2024)
- [Unifying Physics- and Data-Driven Modeling via Novel Causal Spatiotemporal Graph Neural Network for Interpretable Epidemic Forecasting](https://mlsanity.com/paper/2504.05140/) (2025)
- [Accelerating Privacy-Preserving Federated Learning in Large-Scale LEO Satellite Systems](https://mlsanity.com/paper/2509.12222/) (2025)

## Abstract (arXiv)

Federated learning (FL) enables collaborative learning across multiple clients. In most FL work, all clients train a single learning task. However, the recent proliferation of FL applications may increasingly require multiple FL tasks to be trained simultaneously, sharing clients' computing and communication resources, which we call Multiple-Model Federated Learning (MMFL). Current MMFL algorithms use naive average-based client-task allocation schemes that can lead to unfair performance when FL tasks have heterogeneous difficulty levels, e.g., tasks with larger models may need more rounds and data to train. Just as naively allocating resources to generic computing jobs with heterogeneous resource needs can lead to unfair outcomes, naive allocation of clients to FL tasks can lead to unfairness, with some tasks having excessively long training times, or lower converged accuracies. Furthermore, in the FL setting, since clients are typically not paid for their training effort, we face a further challenge that some clients may not even be willing to train some tasks, e.g., due to high computational costs, which may exacerbate unfairness in training outcomes across tasks. We address both challenges by firstly designing FedFairMMFL, a difficulty-aware algorithm that dynamically allocates clients to tasks in each training round. We provide guarantees on airness and FedFairMMFL's convergence rate. We then propose a novel auction design that incentivizes clients to train multiple tasks, so as to fairly distribute clients' training efforts across the tasks. We show how our fairness-based learning and incentive mechanisms impact training convergence and finally evaluate our algorithm with multiple sets of learning tasks on real world datasets.