mlsanityData to 7 Oct 2026

Label Differential Privacy via Aggregation

arXiv 2310.10092 ↗First posted 16 Oct 2023cs.LG stat.MLcited by 1

Not on the map. Why?

Abstract (arXiv)

This paper explores the use of linear aggregation to protect the privacy of sensitive training labels through the concept of label differential privacy (label-DP) while maintaining regression task utility. Our key finding is that weighted linear aggregation of training instances with i.i.d. N(0, 1) weights can achieve (varepsilon, δ)-label-DP with m = Oleft(n/(log(1/δ))right). Unlike prior methods, our approach relies on the minimum linear regression loss rather than the minimum singular value of the data matrix, resulting in better practical bounds on real datasets. We also examine real-world mechanisms involving disjoint sets or textitbags of instances. We demonstrate that aggregating labels from sub-sampled disjoint k-sized bags using i.i.d. N(0,1) weights achieves (varepsilon,δ)-label-DP with k ≥ Omegaleft(left((1/varepsilon)logleft(1/δright)right)2right). In both scenarios, the optimal linear mse-regressor on the aggregated data approximates the original dataset's optimum with high probability, without needing additive label noise. Furthermore, we show that adding N(0,1) noise to any constant fraction of labels allows for similar label-DP guarantees when aggregating labels over random disjoint bags, while preserving the utility of Lipschitz-bounded neural mse-regression tasks.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 20 of its 52 references that are in the map. First posted on arXiv 16 Oct 2023.

Builds on

20 of 52 references.

References · 20

  1. 2018AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networkscited by 319
  2. 2021Deep Learning with Label Differential Privacycited by 113
  3. 2018Revisiting differentially private linear regression: optimal and adaptive prediction & estimation in unbounded domaincited by 60
  4. 2021Antipodes of Label Differential Privacy: PATE and ALIBIcited by 50
  5. 2014On Learning from Label Proportionscited by 38
  6. 2015Differentially Private Ordinary Least Squarescited by 36
  7. 2019Deep multi-class learning from label proportionscited by 33
  8. 2020Differentially Private Simple Linear Regressioncited by 33
  9. 2022Learning from Label Proportions by Learning with Label Noisecited by 30
  10. 2019Learning from Label Proportions with Generative Adversarial Networkscited by 25
  11. 2017Weakly Supervised Classification in High Energy Physicscited by 21
  12. 2021Label differential privacy via clusteringcited by 20
8 more
  1. 2023Easy Learning from Label Proportionscited by 20
  2. 2022Easy Differentially Private Linear Regressioncited by 19
  3. 2023Polynomial Time and Private Learning of Unbounded Gaussian Mixture Modelscited by 19
  4. 2018Reacting to Variations in Product Demand: An Application for Conversion Rate (CR) Prediction in Sponsored Searchcited by 18
  5. 2022Regression with Label Differential Privacycited by 14
  6. 2023Learning from Aggregated Data: Curated Bags versus Random Bagscited by 11
  7. 2022Challenges and approaches to privacy preserving post-click conversion predictioncited by 10
  8. 2023Learning from Label Proportions: Bootstrapping Supervised Learners via Belief Propagationcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Private Learning with Public Feature Conditioningcited by 0