# Private Approximations of the 2nd-Moment Matrix Using Existing Techniques in Linear Regression

arXiv: https://arxiv.org/abs/1507.00056 · HTML: https://mlsanity.com/paper/1507.00056/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jun 2015.

## Built on by (most cited first, 8 of 8)

- [Accuracy, Interpretability, and Differential Privacy via Explainable Boosting](https://mlsanity.com/paper/2106.09680/) (2021)
- [Distributed Sketching for Randomized Optimization: Exact Characterization, Concentration and Lower Bounds](https://mlsanity.com/paper/2203.09755/) (2022)
- [Shape And Structure Preserving Differential Privacy](https://mlsanity.com/paper/2209.12667/) (2022)
- [Fixed-Budget Differentially Private Best Arm Identification](https://mlsanity.com/paper/2401.09073/) (2024)
- [The Gaussian Mixing Mechanism: Renyi Differential Privacy via Gaussian Sketches](https://mlsanity.com/paper/2505.24603/) (2025)
- [Statistical Privacy Guarantees of Machine Learning Preprocessing Techniques](https://mlsanity.com/paper/2109.02496/) (2021)
- [Federated Linear Bandit Learning via Over-the-Air Computation](https://mlsanity.com/paper/2308.13298/) (2023)
- [Differentially Private High Dimensional Bandits](https://mlsanity.com/paper/2402.03737/) (2024)

## Abstract (arXiv)

We introduce three differentially-private algorithms that approximates the 2nd-moment matrix of the data. These algorithm, which in contrast to existing algorithms output positive-definite matrices, correspond to existing techniques in linear regression literature. Specifically, we discuss the following three techniques. (i) For Ridge Regression, we propose setting the regularization coefficient so that by approximating the solution using Johnson-Lindenstrauss transform we preserve privacy. (ii) We show that adding a small batch of random samples to our data preserves differential privacy. (iii) We show that sampling the 2nd-moment matrix from a Bayesian posterior inverse-Wishart distribution is differentially private provided the prior is set correctly. We also evaluate our techniques experimentally and compare them to the existing "Analyze Gauss" algorithm of Dwork et al.