# Label Differential Privacy via Aggregation

arXiv: https://arxiv.org/abs/2310.10092 · HTML: https://mlsanity.com/paper/2310.10092/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 52 references that are in the map.
- First posted on arXiv 16 Oct 2023.

## Builds on (most cited first, 20 of 20)

- [AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks](https://mlsanity.com/paper/1810.11921/) (2018)
- [Deep Learning with Label Differential Privacy](https://mlsanity.com/paper/2102.06062/) (2021)
- [Revisiting differentially private linear regression: optimal and adaptive prediction & estimation in unbounded domain](https://mlsanity.com/paper/1803.02596/) (2018)
- [Antipodes of Label Differential Privacy: PATE and ALIBI](https://mlsanity.com/paper/2106.03408/) (2021)
- [On Learning from Label Proportions](https://mlsanity.com/paper/1402.5902/) (2014)
- [Differentially Private Ordinary Least Squares](https://mlsanity.com/paper/1507.02482/) (2015)
- [Differentially Private Simple Linear Regression](https://mlsanity.com/paper/2007.05157/) (2020)
- [Deep multi-class learning from label proportions](https://mlsanity.com/paper/1905.12909/) (2019)
- [Learning from Label Proportions by Learning with Label Noise](https://mlsanity.com/paper/2203.02496/) (2022)
- [Learning from Label Proportions with Generative Adversarial Networks](https://mlsanity.com/paper/1909.02180/) (2019)
- [Weakly Supervised Classification in High Energy Physics](https://mlsanity.com/paper/1702.00414/) (2017)
- [Label differential privacy via clustering](https://mlsanity.com/paper/2110.02159/) (2021)
- [Easy Learning from Label Proportions](https://mlsanity.com/paper/2302.03115/) (2023)
- [Easy Differentially Private Linear Regression](https://mlsanity.com/paper/2208.07353/) (2022)
- [Polynomial Time and Private Learning of Unbounded Gaussian Mixture Models](https://mlsanity.com/paper/2303.04288/) (2023)
- [Reacting to Variations in Product Demand: An Application for Conversion Rate (CR) Prediction in Sponsored Search](https://mlsanity.com/paper/1806.08211/) (2018)
- [Regression with Label Differential Privacy](https://mlsanity.com/paper/2212.06074/) (2022)
- [Learning from Aggregated Data: Curated Bags versus Random Bags](https://mlsanity.com/paper/2305.09557/) (2023)
- [Challenges and approaches to privacy preserving post-click conversion prediction](https://mlsanity.com/paper/2201.12666/) (2022)
- [Learning from Label Proportions: Bootstrapping Supervised Learners via Belief Propagation](https://mlsanity.com/paper/2310.08056/) (2023)

## Built on by (most cited first, 1 of 1)

- [Private Learning with Public Feature Conditioning](https://mlsanity.com/paper/2606.18773/) (2026)

## Abstract (arXiv)

This paper explores the use of linear aggregation to protect the privacy of sensitive training labels through the concept of label differential privacy (label-DP) while maintaining regression task utility. Our key finding is that weighted linear aggregation of training instances with i.i.d. N(0, 1) weights can achieve (varepsilon, δ)-label-DP with m = Oleft(n/(log(1/δ))right). Unlike prior methods, our approach relies on the minimum linear regression loss rather than the minimum singular value of the data matrix, resulting in better practical bounds on real datasets. We also examine real-world mechanisms involving disjoint sets or textitbags of instances. We demonstrate that aggregating labels from sub-sampled disjoint k-sized bags using i.i.d. N(0,1) weights achieves (varepsilon,δ)-label-DP with k ≥ Omegaleft(left((1/varepsilon)logleft(1/δright)right)2right). In both scenarios, the optimal linear mse-regressor on the aggregated data approximates the original dataset's optimum with high probability, without needing additive label noise. Furthermore, we show that adding N(0,1) noise to any constant fraction of labels allows for similar label-DP guarantees when aggregating labels over random disjoint bags, while preserving the utility of Lipschitz-bounded neural mse-regression tasks.