# Can R'enyi Accountants Beat Lossless Accountants for DP-SGD?

arXiv: https://arxiv.org/abs/2610.17120 · HTML: https://mlsanity.com/paper/2610.17120/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 45 references that are in the map.
- First posted on arXiv 12 Oct 2026.

## Builds on (most cited first, 29 of 29)

- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Opacus: User-Friendly Differential Privacy Library in PyTorch](https://mlsanity.com/paper/2109.12298/) (2021)
- [Concentrated Differential Privacy: Simplifications, Extensions, and Lower Bounds](https://mlsanity.com/paper/1605.02065/) (2016)
- [Subsampled R'enyi Differential Privacy and Analytical Moments Accountant](https://mlsanity.com/paper/1808.00087/) (2018)
- [R'enyi Differential Privacy of the Sampled Gaussian Mechanism](https://mlsanity.com/paper/1908.10530/) (2019)
- [Numerical Composition of Differential Privacy](https://mlsanity.com/paper/2106.02848/) (2021)
- [Concentrated Differential Privacy](https://mlsanity.com/paper/1603.01887/) (2016)
- [Deep Learning with Gaussian Differential Privacy](https://mlsanity.com/paper/1911.11607/) (2019)
- [Privacy Amplification by Iteration](https://mlsanity.com/paper/1808.06651/) (2018)
- [Optimal Accounting of Differential Privacy via Characteristic Function](https://mlsanity.com/paper/2106.08567/) (2021)
- [Hypothesis Testing Interpretations and Renyi Differential Privacy](https://mlsanity.com/paper/1905.09982/) (2019)
- [Federated Learning of Gboard Language Models with Differential Privacy](https://mlsanity.com/paper/2305.18465/) (2023)
- [Connect the Dots: Tighter Discrete Approximations of Privacy Loss Distributions](https://mlsanity.com/paper/2207.04380/) (2022)
- [Differentially Private Learning Needs Hidden State (Or Much Faster Convergence)](https://mlsanity.com/paper/2203.05363/) (2022)
- [Advancing Differential Privacy: Where We Are Now and Future Directions for Real-World Deployment](https://mlsanity.com/paper/2304.06929/) (2023)
- [Three Variants of Differential Privacy: Lossless Conversion and Applications](https://mlsanity.com/paper/2008.06529/) (2020)
- [Faster Privacy Accounting via Evolving Discretization](https://mlsanity.com/paper/2207.04381/) (2022)
- [Privacy amplification by random allocation](https://mlsanity.com/paper/2502.08202/) (2025)
- [Balls-and-Bins Sampling for DP-SGD](https://mlsanity.com/paper/2412.16802/) (2024)
- [Shifted Interpolation for Differential Privacy](https://mlsanity.com/paper/2403.00278/) (2024)
- [Avoiding Pitfalls for Privacy Accounting of Subsampled Mechanisms under Composition](https://mlsanity.com/paper/2405.20769/) (2024)
- [Differentially Private Stochastic Gradient Descent with Fixed-Size Minibatches: Tighter RDP Guarantees with or without Replacement](https://mlsanity.com/paper/2408.10456/) (2024)
- [Convergent Privacy Loss of Noisy-SGD without Convexity and Smoothness](https://mlsanity.com/paper/2410.01068/) (2024)
- [Privacy of the last iterate in cyclically-sampled DP-SGD on nonconvex composite losses](https://mlsanity.com/paper/2407.05237/) (2024)
- [Publishing Wikipedia usage data with strong privacy guarantees](https://mlsanity.com/paper/2308.16298/) (2023)
- [Optimal conversion from R'enyi Differential Privacy to f-Differential Privacy](https://mlsanity.com/paper/2602.04562/) (2026)
- [JAX-Privacy: A library for differentially private machine learning](https://mlsanity.com/paper/2602.17861/) (2026)
- [Privacy Filters are Captured by Residues: A Characterization of Free Natural Filters and the Cost of Adaptivity](https://mlsanity.com/paper/2602.15815/) (2026)
- [Less Random, More Private: What is the Optimal Subsampling Scheme for DP-SGD?](https://mlsanity.com/paper/2605.07072/) (2026)

## Abstract (arXiv)

Recent frameworks in differential privacy (DP), such as privacy loss distributions and f-DP, provide lossless representations of the privacy leakage of mechanisms. These approaches are often considered to yield tighter (varepsilon, δ)-DP guarantees than those obtained from relaxed notions such as R'enyi DP. But is that always the case? We study this question for differentially private stochastic gradient descent (DP-SGD) with fixed-size batches under replace adjacency. Our key observation is that in this setting, privacy accounting based on R'enyi DP can use a pair of worst-case distributions (i.e., a dominating pair) that differs from the one used in the lossless frameworks. In particular, we characterize the worst-case distributions as two-dimensional Gaussian mixtures and derive an improved R'enyi DP bound for DP-SGD. Numerical and asymptotic comparisons show that across various hyperparameter settings, a R'enyi DP accountant based on the new bound can yield better privacy guarantees than those obtained from the lossless frameworks.