# Differentially Private Meta-Learning

arXiv: https://arxiv.org/abs/1909.05830 · HTML: https://mlsanity.com/paper/1909.05830/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Sep 2019.

## Built on by (most cited first, 30 of 41)

- [A Field Guide to Federated Optimization](https://mlsanity.com/paper/2107.06917/) (2021)
- [Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharing](https://mlsanity.com/paper/2106.04502/) (2021)
- [Privacy-Preserving Machine Learning: Methods, Challenges and Directions](https://mlsanity.com/paper/2108.04417/) (2021)
- [Collaborative Federated Learning For Healthcare: Multi-Modal COVID-19 Diagnosis at the Edge](https://mlsanity.com/paper/2101.07511/) (2021)
- [On Privacy and Personalization in Cross-Silo Federated Learning](https://mlsanity.com/paper/2206.07902/) (2022)
- [Motley: Benchmarking Heterogeneity and Personalization in Federated Learning](https://mlsanity.com/paper/2206.09262/) (2022)
- [Federated f-Differential Privacy](https://mlsanity.com/paper/2102.11158/) (2021)
- [Fine-Tuning Large Language Models with User-Level Differential Privacy](https://mlsanity.com/paper/2407.07737/) (2024)
- [Learning Predictions for Algorithms with Predictions](https://mlsanity.com/paper/2202.09312/) (2022)
- [The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning](https://mlsanity.com/paper/2111.05326/) (2021)
- [Personalization Improves Privacy-Accuracy Tradeoffs in Federated Learning](https://mlsanity.com/paper/2202.05318/) (2022)
- [Federated Learning from Only Unlabeled Data with Class-Conditional-Sharing Clients](https://mlsanity.com/paper/2204.03304/) (2022)
- [Learning-to-learn non-convex piecewise-Lipschitz functions](https://mlsanity.com/paper/2108.08770/) (2021)
- [Efficient Federated Meta-Learning over Multi-Access Wireless Networks](https://mlsanity.com/paper/2108.06453/) (2021)
- [Federated Few-Shot Learning with Adversarial Learning](https://mlsanity.com/paper/2104.00365/) (2021)
- [Private Multi-Task Learning: Formulation and Applications to Federated Learning](https://mlsanity.com/paper/2108.12978/) (2021)
- [Federated learning: Applications, challenges and future directions](https://mlsanity.com/paper/2205.09513/) (2022)
- [A Survey on Intelligent Internet of Things: Applications, Security, Privacy, and Future Directions](https://mlsanity.com/paper/2406.03820/) (2024)
- [FL-Market: Trading Private Models in Federated Learning](https://mlsanity.com/paper/2106.04384/) (2021)
- [Mnemosyne: Learning to Train Transformers with Transformers](https://mlsanity.com/paper/2302.01128/) (2023)
- [FedPower: Privacy-Preserving Distributed Eigenspace Estimation](https://mlsanity.com/paper/2103.00704/) (2021)
- [Holistic Survey of Privacy and Fairness in Machine Learning](https://mlsanity.com/paper/2307.15838/) (2023)
- [PRECAD: Privacy-Preserving and Robust Federated Learning via Crypto-Aided Differential Privacy](https://mlsanity.com/paper/2110.11578/) (2021)
- [Transfer Learning In Differential Privacy's Hybrid-Model](https://mlsanity.com/paper/2201.12018/) (2022)
- [Multi-Task Differential Privacy Under Distribution Skew](https://mlsanity.com/paper/2302.07975/) (2023)
- [Mobilizing Personalized Federated Learning in Infrastructure-Less and Heterogeneous Environments via Random Walk Stochastic ADMM](https://mlsanity.com/paper/2304.12534/) (2023)
- [ExpShield: Safeguarding Web Text from Unauthorized Crawling and LLM Exploitation](https://mlsanity.com/paper/2412.21123/) (2024)
- [On the Practicality of Differential Privacy in Federated Learning by Tuning Iteration Times](https://mlsanity.com/paper/2101.04163/) (2021)
- [A Generative Framework for Personalized Learning and Estimation: Theory, Algorithms, and Privacy](https://mlsanity.com/paper/2207.01771/) (2022)
- [Noise-Aware Differentially Private Regression via Meta-Learning](https://mlsanity.com/paper/2406.08569/) (2024)

## Abstract (arXiv)

Parameter-transfer is a well-known and versatile approach for meta-learning, with applications including few-shot learning, federated learning, and reinforcement learning. However, parameter-transfer algorithms often require sharing models that have been trained on the samples from specific tasks, thus leaving the task-owners susceptible to breaches of privacy. We conduct the first formal study of privacy in this setting and formalize the notion of task-global differential privacy as a practical relaxation of more commonly studied threat models. We then propose a new differentially private algorithm for gradient-based parameter transfer that not only satisfies this privacy requirement but also retains provable transfer learning guarantees in convex settings. Empirically, we apply our analysis to the problems of federated learning with personalization and few-shot classification, showing that allowing the relaxation to task-global privacy from the more commonly studied notion of local privacy leads to dramatically increased performance in recurrent neural language modeling and image classification.