# LoRA and Privacy: When Random Projections Help (and When They Don't)

arXiv: https://arxiv.org/abs/2601.21719 · HTML: https://mlsanity.com/paper/2601.21719/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- First posted on arXiv 29 Jan 2026.

## Abstract (arXiv)

We introduce the (Wishart) projection mechanism, a randomized map of the form S mapsto M f(S) with M ~ Wd(1/r Id, r) and study its differential privacy properties. For vector-valued queries f, we prove non-asymptotic DP guarantees without any additive noise, showing that Wishart randomness alone can suffice. For matrix-valued queries, however, we establish a sharp negative result: in the noise-free setting, the mechanism is not DP, and we demonstrate its vulnerability by implementing a near perfect membership inference attack (AUC > 0.99). We then analyze a noisy variant and prove privacy amplification due to randomness and low rank projection, in both large- and small-rank regimes, yielding stronger privacy guarantees than additive noise alone. Finally, we show that LoRA-style updates are an instance of the matrix-valued mechanism, implying that LoRA is not inherently private despite its built-in randomness, but that low-rank fine-tuning can be more private than full fine-tuning at the same noise level. Preliminary experiments suggest that tighter accounting enables lower noise and improved accuracy in practice.