mlsanityData to 7 Oct 2026

Optimal Unbiased Randomizers for Regression with Label Differential Privacy

arXiv 2312.05659 ↗First posted 9 Dec 2023cs.LG cs.CRcited by 6

Not on the map. Why?

Abstract (arXiv)

We propose a new family of label randomizers for training regression models under the constraint of label differential privacy (DP). In particular, we leverage the trade-offs between bias and variance to construct better label randomizers depending on a privately estimated prior distribution over the labels. We demonstrate that these randomizers achieve state-of-the-art privacy-utility trade-offs on several datasets, highlighting the importance of reducing bias when training neural networks with label DP. We also provide theoretical results shedding light on the structural properties of the optimal unbiased randomizers.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 23 of its 42 references that are in the map. First posted on arXiv 9 Dec 2023.

Builds on

23 of 42 references.

References · 23

  1. 2016Deep Learning with Differential Privacycited by 2,682
  2. 2021Differentially Private Fine-tuning of Language Modelscited by 392
  3. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 347
  4. 2018Scalable Private Learning with PATEcited by 266
  5. 2022Unlocking High-Accuracy Differentially Private Image Classification through Scalecited by 218
  6. 2021Deep Learning with Label Differential Privacycited by 113
  7. 2021Hyperparameter Tuning with Renyi Differential Privacycited by 104
  8. 2021Label Leakage and Protection in Two-party Split Learningcited by 87
  9. 2022Toward Training at ImageNet Scale with Differential Privacycited by 87
  10. 2020An Equivalence Between Private Classification and Online Predictioncited by 63
  11. 2014Private Learning and Sanitization: Pure vs. Approximate Differential Privacycited by 62
  12. 2018Revisiting differentially private linear regression: optimal and adaptive prediction & estimation in unbounded domaincited by 60
11 more
  1. 2022Exploring the Limits of Differentially Private Deep Learning with Group-wise Clippingcited by 52
  2. 2022TAN Without a Burn: Scaling Laws of DP-SGDcited by 51
  3. 2021Antipodes of Label Differential Privacy: PATE and ALIBIcited by 50
  4. 2020Differentially Private Simple Linear Regressioncited by 33
  5. 2017Heavy Hitters and the Structure of Local Privacycited by 20
  6. 2019Data Poisoning Attacks to Local Differential Privacy Protocolscited by 20
  7. 2021Label differential privacy via clusteringcited by 20
  8. 2018Reacting to Variations in Product Demand: An Application for Conversion Rate (CR) Prediction in Sponsored Searchcited by 18
  9. 2021Differentially Private Aggregation in the Shuffle Model: Almost Central Accuracy in Almost a Single Messagecited by 18
  10. 2022Regression with Label Differential Privacycited by 14
  11. 2022Differentially Private Learning with Margin Guaranteescited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 1; 2026: 2.

6 papers.

  1. 2024Retraining with Predicted Hard Labels Provably Increases Model Accuracycited by 4
  2. 2024Locally Private Estimation with Public Featurescited by 3
  3. 2024Enhancing Learning with Label Differential Privacy by Vector Approximationcited by 2
  4. 2026RPWithPrior: Label Differential Privacy in Regressioncited by 1
  5. 2025On Theoretical Limits of Learning with Label Differential Privacycited by 0
  6. 2026BlockRR: A Unified Framework of RR-type Algorithms for Label Differential Privacycited by 0