Data to 14 Oct 2026

Deep Learning with Label Differential Privacy

arXiv 2102.06062 ↗First posted 11 Feb 2021cs.LG cs.DScited by 117

Not on the map. Why?

Abstract (arXiv)

The Randomized Response (RR) algorithm is a classical technique to improve robustness in survey aggregation, and has been widely adopted in applications with differential privacy guarantees. We propose a novel algorithm, Randomized Response with Prior (RRWithPrior), which can provide more accurate results while maintaining the same level of privacy guaranteed by RR. We then apply RRWithPrior to learn neural networks with label differential privacy (LabelDP), and show that when only the label needs to be protected, the model performance can be significantly improved over the previous state-of-the-art private baselines. Moreover, we study different ways to obtain priors, which when used with RRWithPrior can additionally improve the model performance, further reducing the accuracy gap between private and non-private models. We complement the empirical results with theoretical analysis showing that LabelDP is provably easier than protecting both the inputs and labels.

Key facts. Cited by 117 papers in the map (data through 14 Oct 2026). Builds on 54 of its 109 references that are in the map. First posted on arXiv 11 Feb 2021.

Builds on

54 of 109 references.

References · 54

2015Deep Residual Learning for Image Recognitioncited by 52,096
2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
2014Going Deeper with Convolutionscited by 4,241
2016Deep Learning with Differential Privacycited by 2,728
2016Identity Mappings in Deep Residual Networkscited by 1,965
2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,628
2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
2017Neural Collaborative Filteringcited by 1,111

38 more

2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,059
2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 997
2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 582
2018Deep Learning for Classical Japanese Literaturecited by 422
2019How does Disagreement Help Generalization against Label Corruption?cited by 390
2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 348
2014RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Responsecited by 272
2018Scalable Private Learning with PATEcited by 269
2020Identifying Mislabeled Data using the Area Under the Margin Rankingcited by 261
2020Normalized Loss Functions for Deep Learning with Noisy Labelscited by 257
2020Differentially Private Learning Needs Better Features (or Much More Data)cited by 235
2019Understanding and Utilizing Deep Neural Networks Trained with Noisy Labelscited by 209
2020Does label smoothing mitigate label noise?cited by 204
2019Private Stochastic Convex Optimization with Optimal Ratescited by 186
2018Amplification by Shuffling: From Local to Central Differential Privacy via Anonymitycited by 164
2019SELF: Learning to Filter Noisy Labels with Self-Ensemblingcited by 164
2020Private Stochastic Convex Optimization: Optimal Rates in Linear Timecited by 153
2017Collecting Telemetry Data Privatelycited by 149
2018Differentially Private Empirical Risk Minimization Revisited: Faster and More Generalcited by 128
2020Tempered Sigmoid Activations for Deep Learning with Differential Privacycited by 122
2018Distributed Differential Privacy via Shufflingcited by 115
2019Deep Learning with Gaussian Differential Privacycited by 114
2016Minimax Optimal Procedures for Locally Private Estimationcited by 107
2019BackPACK: Packing more into backpropcited by 78
2018Private PAC learning implies finite Littlestone dimensioncited by 74
2020Error-Bounded Correction of Noisy Labelscited by 74
2019Differentially Private Model Publishing for Deep Learningcited by 71
2014Private Learning and Sanitization: Pure vs. Approximate Differential Privacycited by 63
2017Efficient Private ERM for Smooth Objectivescited by 61
2016Discrete Distribution Estimation under Local Privacycited by 59
2020Individual Privacy Accounting via a Renyi Filtercited by 59
2020Enabling Fast Differentially Private SGD via Just-in-Time Compilation and Vectorizationcited by 58
2015Between Pure and Approximate Differential Privacycited by 56
2015Efficient Per-Example Gradient Computationscited by 42
2019How does Early Stopping Help Generalization against Label Noise?cited by 41
2020Improving Generalization by Controlling Label-Noise Information in Neural Network Weightscited by 35
2020Improving Deep Learning with Differential Privacy using Gradient Encoding and Denoisingcited by 30
2013Local Privacy and Minimax Bounds: Sharp Rates for Probability Estimationcited by 19

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 5; 2022: 18; 2023: 34; 2024: 23; 2025: 21; 2026: 16.

117 papers.

Sort:

By area:

2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 153
2022Measuring Forgetting of Memorized Training Examplescited by 118
2023On Provable Copyright Protection for Generative Modelscited by 98
2021Label Leakage and Protection in Two-party Split Learningcited by 90
2023DP-Forward: Fine-tuning and Inference on Language Models with Differential Privacy in Forward Passcited by 67
2021Antipodes of Label Differential Privacy: PATE and ALIBIcited by 52
2022Just Fine-tune Twice: Selective Differential Privacy for Large Language Modelscited by 32
2022Label Leakage and Protection from Forward Embedding in Vertical Federated Learningcited by 27
2022Differentially Private Label Protection in Split Learningcited by 26
2023Differentially Private Attention Computationcited by 25
2021Label differential privacy via clusteringcited by 20
2022How unfair is private learning ?cited by 18

38 more

2022Weak Proxies are Sufficient and Preferable for Fairness with Missing Sensitive Attributescited by 17
2022Vertical Federated Learning: A Structured Literature Reviewcited by 17
2021ExPLoit: Extracting Private Labels in Split Learningcited by 15
2022Similarity-based Label Inference Attack against Training and Inference of Split Learningcited by 15
2022Regression with Label Differential Privacycited by 15
2025Towards User-level Private Reinforcement Learning with Human Feedbackcited by 14
2023Communication-Efficient Vertical Federated Learning with Limited Overlapping Samplescited by 13
2023DPMLBench: Holistic Evaluation of Differentially Private Machine Learningcited by 13
2023MergeSFL: Split Federated Learning with Feature Merging and Batch Size Regulationcited by 13
2023Differentially Private Reward Estimation with Preference Feedbackcited by 12
2022Does Label Differential Privacy Prevent Label Inference Attacks?cited by 11
2023Learning from Aggregated Data: Curated Bags versus Random Bagscited by 11
2022PADDLES: Phase-Amplitude Spectrum Disentangled Early Stopping for Learning with Noisy Labelscited by 10
2023Simple Binary Hypothesis Testing under Local Differential Privacy and Communication Constraintscited by 10
2022GeoPointGAN: Synthetic Spatial Data with Local Label Differential Privacycited by 9
2023Differentially Private Video Activity Recognitioncited by 9
2024Laboratory-Scale AI: Open-Weight Models are Competitive with ChatGPT Even in Low-Resource Settingscited by 9
2023Label Inference Attack against Split Learning under Regression Settingcited by 8
2023Learning From Biased Soft Labelscited by 8
2023VFLAIR: A Research Library and Benchmark for Vertical Federated Learningcited by 8
2025A Unified Theoretical Analysis of Private and Robust Offline Alignment: from RLHF to DPOcited by 8
2022Differentially Private Learning with Margin Guaranteescited by 7
2022DPAUC: Differentially Private AUC Computation in Federated Learningcited by 7
2023Robust and IP-Protecting Vertical Federated Learning against Unexpected Quitting of Partiescited by 7
2024A Survey of Privacy Threats and Defense in Vertical Federated Learning: From Model Life Cycle Perspectivecited by 7
2024DPAdapter: Improving Differentially Private Deep Learning through Noise Tolerance Pre-trainingcited by 7
2022Private Ad Modeling with DP-SGDcited by 6
2023Eliminating Label Leakage in Tree-Based Vertical Federated Learningcited by 6
2023Private Learning with Public Featurescited by 6
2023Optimal Unbiased Randomizers for Regression with Label Differential Privacycited by 6
2024Training Differentially Private Ad Prediction Models with Semi-Sensitive Featurescited by 6
2023Little is Enough: Boosting Privacy by Sharing Only Hard Labels in Federated Semi-Supervised Learningcited by 5
2024Locally Differentially Private In-Context Learningcited by 5
2024Rethinking the impact of noisy labels in graph classification: A utility and privacy perspectivecited by 5
2024Retraining with Predicted Hard Labels Provably Increases Model Accuracycited by 5
2025Conformal Prediction of Classifiers with Many Classes based on Noisy Labelscited by 5
2025The Hitchhiker's Guide to Efficient, End-to-End, and Tight DP Auditingcited by 5
2022FEL: High Capacity Learning for Recommendation and Ranking via Federated Ensemble Learningcited by 4

50 most cited of 117 All 117 (JSON)