# Integrating Feature Correlation in Differential Privacy with Applications in DP-ERM

arXiv: https://arxiv.org/abs/2605.03945 · HTML: https://mlsanity.com/paper/2605.03945/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 47 references that are in the map.
- First posted on arXiv 5 May 2026.

## Builds on (most cited first, 25 of 25)

- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Condition](https://mlsanity.com/paper/1608.04636/) (2016)
- [Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergences](https://mlsanity.com/paper/1807.01647/) (2018)
- [Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Learning](https://mlsanity.com/paper/1609.06570/) (2016)
- [Differentially Private Empirical Risk Minimization Revisited: Faster and More General](https://mlsanity.com/paper/1802.05251/) (2018)
- [Deep Learning with Label Differential Privacy](https://mlsanity.com/paper/2102.06062/) (2021)
- [Bypassing the Ambient Dimension: Private SGD with Gradient Subspace Identification](https://mlsanity.com/paper/2007.03813/) (2020)
- [Fingerprinting Codes and the Price of Approximate Differential Privacy](https://mlsanity.com/paper/1311.3158/) (2013)
- [Selective Differential Privacy for Language Modeling](https://mlsanity.com/paper/2108.12944/) (2021)
- [Private Adaptive Gradient Methods for Convex Optimization](https://mlsanity.com/paper/2106.13756/) (2021)
- [Limits of Private Learning with Access to Public Data](https://mlsanity.com/paper/1910.11519/) (2019)
- [Private Adaptive Optimization with Side Information](https://mlsanity.com/paper/2202.05963/) (2022)
- [Empirical Risk Minimization in Non-interactive Local Differential Privacy: Efficiency and High Dimensional Case](https://mlsanity.com/paper/1802.04085/) (2018)
- [Attribute Privacy: Framework and Mechanisms](https://mlsanity.com/paper/2009.04013/) (2020)
- [Differentially Private Sliced Wasserstein Distance](https://mlsanity.com/paper/2107.01848/) (2021)
- [Context-Aware Local Differential Privacy](https://mlsanity.com/paper/1911.00038/) (2019)
- [Pufferfish Privacy Mechanisms for Correlated Data](https://mlsanity.com/paper/1603.03977/) (2016)
- [Optimal Differentially Private Model Training with Public Data](https://mlsanity.com/paper/2306.15056/) (2023)
- [Correlated Data in Differential Privacy: Definition and Analysis](https://mlsanity.com/paper/2008.00180/) (2020)
- [Training Differentially Private Ad Prediction Models with Semi-Sensitive Features](https://mlsanity.com/paper/2401.15246/) (2024)
- [Profile-Based Privacy for Locally Private Computations](https://mlsanity.com/paper/1903.09084/) (2019)
- [Task-aware Privacy Preservation for Multi-dimensional Data](https://mlsanity.com/paper/2110.02329/) (2021)
- [Enhancing Feature-Specific Data Protection via Bayesian Coordinate Differential Privacy](https://mlsanity.com/paper/2410.18404/) (2024)
- [Managing Correlations in Data and Privacy Demand](https://mlsanity.com/paper/2509.02856/) (2025)
- [On Convex Optimization with Semi-Sensitive Features](https://mlsanity.com/paper/2406.19040/) (2024)

## Abstract (arXiv)

Standard differential privacy imposes uniform privacy constraints across all features, overlooking the inherent distinction between sensitive and insensitive features in practice. In this paper, we introduce a relaxed definition of differential privacy that accounts for such heterogeneity, allowing certain features to be treated as insensitive even when correlated with sensitive ones. We propose a correlation-aware framework, textsfCorrDP, which relaxes privacy for insensitive features while accounting for their correlations with sensitive features, with the correlations quantified using total variation distance. We design algorithms for differentially private empirical risk minimization (DP-ERM) under the textsfCorrDP framework, incorporating distance-dependent noise into gradients for improved theoretical utility guarantees. When the correlation distance is unknown, we estimate it from the dataset and show that it achieves a comparable privacy-utility guarantee. We perform experiments on synthetic and real-world datasets and show that textsfCorrDP-based DP-ERM algorithms consistently outperform the standard DP framework in the presence of insensitive features.