# SoftAdaClip: A Smooth Clipping Strategy for Fair and Private Model Training

arXiv: https://arxiv.org/abs/2510.01447 · HTML: https://mlsanity.com/paper/2510.01447/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 44 references that are in the map.
- First posted on arXiv 1 Oct 2025.

## Builds on (most cited first, 26 of 26)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [Large Language Models Can Be Strong Differentially Private Learners](https://mlsanity.com/paper/2110.05679/) (2021)
- [Opacus: User-Friendly Differential Privacy Library in PyTorch](https://mlsanity.com/paper/2109.12298/) (2021)
- [Differential Privacy Has Disparate Impact on Model Accuracy](https://mlsanity.com/paper/1905.12101/) (2019)
- [Subsampled R'enyi Differential Privacy and Analytical Moments Accountant](https://mlsanity.com/paper/1808.00087/) (2018)
- [Differentially Private Learning with Adaptive Clipping](https://mlsanity.com/paper/1905.03871/) (2019)
- [Numerical Composition of Differential Privacy](https://mlsanity.com/paper/2106.02848/) (2021)
- [A survey on datasets for fairness-aware machine learning](https://mlsanity.com/paper/2110.00530/) (2021)
- [Deep Learning with Gaussian Differential Privacy](https://mlsanity.com/paper/1911.11607/) (2019)
- [Automatic Clipping: Differentially Private Deep Learning Made Easier and Stronger](https://mlsanity.com/paper/2206.07136/) (2022)
- [Optimal Accounting of Differential Privacy via Characteristic Function](https://mlsanity.com/paper/2106.08567/) (2021)
- [Neither Private Nor Fair: Impact of Data Imbalance on Utility and Fairness in Differential Privacy](https://mlsanity.com/paper/2009.06389/) (2020)
- [Fair Decision Making using Privacy-Protected Data](https://mlsanity.com/paper/1905.12744/) (2019)
- [Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration](https://mlsanity.com/paper/2102.04110/) (2021)
- [Large Scale Transfer Learning for Differentially Private Image Classification](https://mlsanity.com/paper/2205.02973/) (2022)
- [Chasing Your Long Tails: Differentially Private Prediction in Health Care Settings](https://mlsanity.com/paper/2010.06667/) (2020)
- [Federated Learning for Privacy Preservation in Smart Healthcare Systems: A Comprehensive Survey](https://mlsanity.com/paper/2203.09702/) (2022)
- [Disparate Impact in Differential Privacy from Gradient Misalignment](https://mlsanity.com/paper/2206.07737/) (2022)
- [Differentially Private Learning with Per-Sample Adaptive Clipping](https://mlsanity.com/paper/2212.00328/) (2022)
- [Removing Disparate Impact of Differentially Private Stochastic Gradient Descent on Model Accuracy](https://mlsanity.com/paper/2003.03699/) (2020)

## Built on by (most cited first, 1 of 1)

- [Differential Privacy in Two-Layer Networks: How DP-SGD Harms Fairness and Robustness](https://mlsanity.com/paper/2603.04881/) (2026)

## Abstract (arXiv)

Differential privacy (DP) provides strong protection for sensitive data, but often reduces model performance and fairness, especially for underrepresented groups. One major reason is gradient clipping in DP-SGD, which can disproportionately suppress learning signals for minority subpopulations. Although adaptive clipping can enhance utility, it still relies on uniform hard clipping, which may restrict fairness. To address this, we introduce SoftAdaClip, a differentially private training method that replaces hard clipping with a smooth, tanh-based transformation to preserve relative gradient magnitudes while bounding sensitivity. We evaluate SoftAdaClip on various datasets, including MIMIC-III (clinical text), GOSSIS-eICU (structured healthcare), and Adult Income (tabular data). Our results show that SoftAdaClip reduces subgroup disparities by up to 87% compared to DP-SGD and up to 48% compared to Adaptive-DPSGD, and these reductions in subgroup disparities are statistically significant. These findings underscore the importance of integrating smooth transformations with adaptive mechanisms to achieve fair and private model training.