mlsanityData to 14 Oct 2026

Private Multi-Task Learning: Formulation and Applications to Federated Learning

arXiv 2108.12978 ↗First posted 30 Aug 2021cs.LG cs.CRcited by 11

Not on the map. Why?

Abstract (arXiv)

Many problems in machine learning rely on multi-task learning (MTL), in which the goal is to solve multiple related machine learning tasks simultaneously. MTL is particularly relevant for privacy-sensitive applications in areas such as healthcare, finance, and IoT computing, where sensitive data from multiple, varied sources are shared for the purpose of learning. In this work, we formalize notions of client-level privacy for MTL via joint differential privacy (JDP), a relaxation of differential privacy for mechanism design and distributed optimization. We then propose an algorithm for mean-regularized MTL, an objective commonly used for applications in personalized federated learning, subject to JDP. We analyze our objective and solver, providing certifiable guarantees on both privacy and utility. Empirically, we find that our method provides improved privacy/utility trade-offs relative to global baselines across common federated learning benchmarks.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 38 of its 60 references that are in the map. First posted on arXiv 30 Aug 2021.

Builds on

38 of 60 references.

References · 38

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  4. 2019Advances and Open Problems in Federated Learningcited by 3,046
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  6. 2016Deep Learning with Differential Privacycited by 2,728
  7. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  8. 2017A Survey on Multi-Task Learningcited by 840
  9. 2018LEAF: A Benchmark for Federated Settingscited by 792
  10. 2017Federated Multi-Task Learningcited by 672
  11. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  12. 2017Renyi Differential Privacycited by 650
26 more
  1. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  2. 2020An Efficient Framework for Clustered Federated Learningcited by 511
  3. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 503
  4. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 459
  5. 2020Adaptive Personalized Federated Learningcited by 358
  6. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 325
  7. 2017Multitask learning and benchmarking with clinical time series datacited by 290
  8. 2020Federated Learning of a Mixture of Global and Local Modelscited by 210
  9. 2016Practical Secure Aggregation for Federated Learning on User-Held Datacited by 178
  10. 2020Salvaging Federated Learning by Local Adaptationcited by 142
  11. 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 128
  12. 2020Lower Bounds and Optimal Algorithms for Personalized Federated Learningcited by 123
  13. 2021Learning with User-Level Privacycited by 72
  14. 2019Differentially Private Meta-Learningcited by 41
  15. 2021Private Federated Learning Without a Trusted Server: Optimal Algorithms for Convex Lossescited by 30
  16. 2013Private Matchings and Allocationscited by 23
  17. 2019Federated Learning with Autotuned Communication-Efficient Secure Aggregationcited by 23
  18. 2020Federated Residual Learningcited by 23
  19. 2018Learning Tasks for Multitask Learning: Heterogenous Patient Populations in the ICUcited by 22
  20. 2016Asynchronous Multi-Task Learningcited by 8
  21. 2020A Theoretical Perspective on Differentially Private Federated Multi-task Learningcited by 8
  22. 2016Distributed Multi-Task Relationship Learningcited by 7
  23. 2014Privacy and Truthful Equilibrium Selection for Aggregative Gamescited by 6
  24. 2013Asymptotically Truthful Equilibrium Selection in Large Congestion Gamescited by 5
  25. 2014Jointly Private Convex Programmingcited by 4
  26. 2014Approximately Stable, School Optimal, and Student-Truthful Many-to-One Matchings (via Differential Privacy)cited by 1

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 4; 2024: 0; 2025: 3.

11 papers.

  1. 2022On Privacy and Personalization in Cross-Silo Federated Learningcited by 36
  2. 2022Personalization Improves Privacy-Accuracy Tradeoffs in Federated Learningcited by 23
  3. 2023Share Your Representation Only: Guaranteed Improvement of the Privacy-Utility Tradeoff in Federated Learningcited by 13
  4. 2022Straggler-Resilient Personalized Federated Learningcited by 9
  5. 2025Privacy-Preserving Personalized Federated Prompt Learning for Multimodal Large Language Modelscited by 5
  6. 2022Training a Tokenizer for Free with Private Federated Learningcited by 4
  7. 2023Multi-Task Differential Privacy Under Distribution Skewcited by 3
  8. 2023Collaborative Learning From Distributed Data With Differentially Private Synthetic Twin Datacited by 1
  9. 2023Domain Private Transformers for Multi-Domain Dialog Systemscited by 0
  10. 2025ColNet: Collaborative Optimization in Decentralized Federated Multi-task Learning Systemscited by 0
  11. 2025Black-Box Privacy Attacks on Shared Representations in Multitask Learningcited by 0