# Federated and differentially private estimation of KL divergence

arXiv: https://arxiv.org/abs/2411.16478 · HTML: https://mlsanity.com/paper/2411.16478/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 31 references that are in the map.
- First posted on arXiv 25 Nov 2024.

## Builds on (most cited first, 8 of 8)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [Prio: Private, Robust, and Scalable Computation of Aggregate Statistics](https://mlsanity.com/paper/1703.06255/) (2017)
- [Differentially Private Markov Chain Monte Carlo](https://mlsanity.com/paper/1901.10275/) (2019)
- [Privately Customizing Prefinetuning to Better Match User Data in Federated Learning](https://mlsanity.com/paper/2302.09042/) (2023)
- [Differentially Private Bayesian Learning on Distributed Data](https://mlsanity.com/paper/1703.01106/) (2017)

## Abstract (arXiv)

Measuring distribution drifts is a key task in managing distributed, sensitive data, as it underpins a wide range of federated learning and analytics applications. In many practical settings, however, directly sharing such information is either undesirable (e.g., due to privacy concerns) or infeasible (e.g., due to high communication costs). In this work, we present FedPriKL, a novel method for estimating the KL divergence of data across federated computational models under differential privacy (DP) guarantees. We establish its theoretical properties, showing that FedPriKL is unbiased with low and bounded sensitivity and variance, thereby ensuring strong utility under DP. In addition, we present an empirical study that explores parameter choices to optimize accuracy while minimizing communication overhead. Our experiments demonstrate that FedPriKL achieves accuracy comparable to a similar sampling-based non-private estimator, while delivering greater stability and accuracy than a baseline variant in which users perturb their data locally to obtain privacy guarantees.