SoftAdaClip: A Smooth Clipping Strategy for Fair and Private Model Training
Not on the map. Why?
Abstract (arXiv)
Differential privacy (DP) provides strong protection for sensitive data, but often reduces model performance and fairness, especially for underrepresented groups. One major reason is gradient clipping in DP-SGD, which can disproportionately suppress learning signals for minority subpopulations. Although adaptive clipping can enhance utility, it still relies on uniform hard clipping, which may restrict fairness. To address this, we introduce SoftAdaClip, a differentially private training method that replaces hard clipping with a smooth, tanh-based transformation to preserve relative gradient magnitudes while bounding sensitivity. We evaluate SoftAdaClip on various datasets, including MIMIC-III (clinical text), GOSSIS-eICU (structured healthcare), and Adult Income (tabular data). Our results show that SoftAdaClip reduces subgroup disparities by up to 87% compared to DP-SGD and up to 48% compared to Adaptive-DPSGD, and these reductions in subgroup disparities are statistically significant. These findings underscore the importance of integrating smooth transformations with adaptive mechanisms to achieve fair and private model training.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 26 of its 44 references that are in the map. First posted on arXiv 1 Oct 2025.
Builds on
26 of 44 references.
References · 26
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2016Deep Learning with Differential Privacycited by 2,682
- 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
- 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
- 2020Extracting Training Data from Large Language Modelscited by 2,144
- 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
- 2017Renyi Differential Privacycited by 643
- 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
- 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 375
- 2019Differential Privacy Has Disparate Impact on Model Accuracycited by 316
- 2018Subsampled R'enyi Differential Privacy and Analytical Moments Accountantcited by 234
- 2019Differentially Private Learning with Adaptive Clippingcited by 218
14 more
- 2021Numerical Composition of Differential Privacycited by 174
- 2021A survey on datasets for fairness-aware machine learningcited by 157
- 2019Deep Learning with Gaussian Differential Privacycited by 112
- 2022Automatic Clipping: Differentially Private Deep Learning Made Easier and Strongercited by 87
- 2021Optimal Accounting of Differential Privacy via Characteristic Functioncited by 83
- 2020Neither Private Nor Fair: Impact of Data Imbalance on Utility and Fairness in Differential Privacycited by 61
- 2019Fair Decision Making using Privacy-Protected Datacited by 58
- 2021Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integrationcited by 43
- 2022Large Scale Transfer Learning for Differentially Private Image Classificationcited by 40
- 2020Chasing Your Long Tails: Differentially Private Prediction in Health Care Settingscited by 37
- 2022Federated Learning for Privacy Preservation in Smart Healthcare Systems: A Comprehensive Surveycited by 31
- 2022Disparate Impact in Differential Privacy from Gradient Misalignmentcited by 31
- 2022Differentially Private Learning with Per-Sample Adaptive Clippingcited by 21
- 2020Removing Disparate Impact of Differentially Private Stochastic Gradient Descent on Model Accuracycited by 13
Built on by
Citing papers per year: 2026: 1.
1 paper.