# Regression with Label Differential Privacy

arXiv: https://arxiv.org/abs/2212.06074 · HTML: https://mlsanity.com/paper/2212.06074/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 42 references that are in the map.
- First posted on arXiv 12 Dec 2022.

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Opacus: User-Friendly Differential Privacy Library in PyTorch](https://mlsanity.com/paper/2109.12298/) (2021)
- [Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data](https://mlsanity.com/paper/1610.05755/) (2016)
- [Scalable Private Learning with PATE](https://mlsanity.com/paper/1802.08908/) (2018)
- [Deep Learning with Label Differential Privacy](https://mlsanity.com/paper/2102.06062/) (2021)
- [Label Leakage and Protection in Two-party Split Learning](https://mlsanity.com/paper/2102.08504/) (2021)
- [Private Learning and Sanitization: Pure vs. Approximate Differential Privacy](https://mlsanity.com/paper/1407.2674/) (2014)
- [Antipodes of Label Differential Privacy: PATE and ALIBI](https://mlsanity.com/paper/2106.03408/) (2021)
- [Data Poisoning Attacks to Local Differential Privacy Protocols](https://mlsanity.com/paper/1911.02046/) (2019)
- [Label differential privacy via clustering](https://mlsanity.com/paper/2110.02159/) (2021)
- [Reacting to Variations in Product Demand: An Application for Conversion Rate (CR) Prediction in Sponsored Search](https://mlsanity.com/paper/1806.08211/) (2018)
- [Differentially Private Aggregation in the Shuffle Model: Almost Central Accuracy in Almost a Single Message](https://mlsanity.com/paper/2109.13158/) (2021)
- [Handling many conversions per click in modeling delayed feedback](https://mlsanity.com/paper/2101.02284/) (2021)

## Built on by (most cited first, 15 of 15)

- [Private Learning with Public Features](https://mlsanity.com/paper/2310.15454/) (2023)
- [Optimal Unbiased Randomizers for Regression with Label Differential Privacy](https://mlsanity.com/paper/2312.05659/) (2023)
- [Retraining with Predicted Hard Labels Provably Increases Model Accuracy](https://mlsanity.com/paper/2406.11206/) (2024)
- [Exactly Minimax-Optimal Locally Differentially Private Sampling](https://mlsanity.com/paper/2410.22699/) (2024)
- [Locally Private Estimation with Public Features](https://mlsanity.com/paper/2405.13481/) (2024)
- [Enhancing Learning with Label Differential Privacy by Vector Approximation](https://mlsanity.com/paper/2405.15150/) (2024)
- [Label Differential Privacy via Aggregation](https://mlsanity.com/paper/2310.10092/) (2023)
- [On Convex Optimization with Semi-Sensitive Features](https://mlsanity.com/paper/2406.19040/) (2024)
- [On Theoretical Limits of Learning with Label Differential Privacy](https://mlsanity.com/paper/2502.14309/) (2025)
- [RPWithPrior: Label Differential Privacy in Regression](https://mlsanity.com/paper/2601.22625/) (2026)
- [Aggregating Data for Optimal and Private Learning](https://mlsanity.com/paper/2411.19045/) (2024)
- [BlockRR: A Unified Framework of RR-type Algorithms for Label Differential Privacy](https://mlsanity.com/paper/2602.03277/) (2026)
- [Private Learning with Public Feature Conditioning](https://mlsanity.com/paper/2606.18773/) (2026)
- [End-to-End Differential Privacy in Training Deep Neural Network Classifiers](https://mlsanity.com/paper/2607.19580/) (2026)
- [High-Dimensional Asymptotics and Dataset Selection for Private Transfer Learning](https://mlsanity.com/paper/2610.02578/) (2026)

## Abstract (arXiv)

We study the task of training regression models with the guarantee of label differential privacy (DP). Based on a global prior distribution on label values, which could be obtained privately, we derive a label DP randomization mechanism that is optimal under a given regression loss function. We prove that the optimal mechanism takes the form of a "randomized response on bins", and propose an efficient algorithm for finding the optimal bin values. We carry out a thorough experimental evaluation on several datasets demonstrating the efficacy of our algorithm.