# Noise-Aware Differentially Private Regression via Meta-Learning

arXiv: https://arxiv.org/abs/2406.08569 · HTML: https://mlsanity.com/paper/2406.08569/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 41 references that are in the map.
- First posted on arXiv 12 Jun 2024.

## Builds on (most cited first, 28 of 28)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [Large Language Models Can Be Strong Differentially Private Learners](https://mlsanity.com/paper/2110.05679/) (2021)
- [Differentially Private Fine-tuning of Language Models](https://mlsanity.com/paper/2110.06500/) (2021)
- [Opacus: User-Friendly Differential Privacy Library in PyTorch](https://mlsanity.com/paper/2109.12298/) (2021)
- [Neural Processes](https://mlsanity.com/paper/1807.01622/) (2018)
- [Unlocking High-Accuracy Differentially Private Image Classification through Scale](https://mlsanity.com/paper/2204.13650/) (2022)
- [Convolutional Conditional Neural Processes](https://mlsanity.com/paper/1910.13556/) (2019)
- [Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining](https://mlsanity.com/paper/2212.06470/) (2022)
- [Meta-Learning Stationary Stochastic Process Prediction with Convolutional Neural Processes](https://mlsanity.com/paper/2007.01332/) (2020)
- [Understanding Probabilistic Sparse Gaussian Process Approximations](https://mlsanity.com/paper/1606.04820/) (2016)
- [Neural Diffusion Processes](https://mlsanity.com/paper/2206.03992/) (2022)
- [Differentially Private Meta-Learning](https://mlsanity.com/paper/1909.05830/) (2019)
- [Autoregressive Conditional Neural Processes](https://mlsanity.com/paper/2303.14468/) (2023)
- [Practical Conditional Neural Processes Via Tractable Dependent Predictions](https://mlsanity.com/paper/2203.08775/) (2022)
- [Differentially Private Image Classification by Learning Priors from Random Processes](https://mlsanity.com/paper/2306.06076/) (2023)
- [The Bernstein Mechanism: Function Release under Differential Privacy](https://mlsanity.com/paper/1507.04499/) (2015)
- [Practical Equivariances via Relational Conditional Neural Processes](https://mlsanity.com/paper/2306.10915/) (2023)
- [On the Efficacy of Differentially Private Few-shot Image Classification](https://mlsanity.com/paper/2302.01190/) (2023)

## Built on by (most cited first, 2 of 2)

- [Gaussian DP for Reporting Differential Privacy Guarantees in Machine Learning](https://mlsanity.com/paper/2503.10945/) (2025)
- [Escaping Iterative Parameter-Space Noise: Differentially Private Learning with a Hypernetwork](https://mlsanity.com/paper/2606.26772/) (2026)

## Abstract (arXiv)

Many high-stakes applications require machine learning models that protect user privacy and provide well-calibrated, accurate predictions. While Differential Privacy (DP) is the gold standard for protecting user privacy, standard DP mechanisms typically significantly impair performance. One approach to mitigating this issue is pre-training models on simulated data before DP learning on the private data. In this work we go a step further, using simulated data to train a meta-learning model that combines the Convolutional Conditional Neural Process (ConvCNP) with an improved functional DP mechanism of Hall et al. [2013] yielding the DPConvCNP. DPConvCNP learns from simulated data how to map private data to a DP predictive model in one forward pass, and then provides accurate, well-calibrated predictions. We compare DPConvCNP with a DP Gaussian Process (GP) baseline with carefully tuned hyperparameters. The DPConvCNP outperforms the GP baseline, especially on non-Gaussian data, yet is much faster at test time and requires less tuning.