# Bounds on Lp errors in density ratio estimation via f-divergence loss functions

arXiv: https://arxiv.org/abs/2410.01516 · HTML: https://mlsanity.com/paper/2410.01516/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 34 references that are in the map.
- First posted on arXiv 2 Oct 2024.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [MINE: Mutual Information Neural Estimation](https://mlsanity.com/paper/1801.04062/) (2018)
- [On Variational Bounds of Mutual Information](https://mlsanity.com/paper/1905.06922/) (2019)
- [f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization](https://mlsanity.com/paper/1606.00709/) (2016)
- [Towards Principled Methods for Training Generative Adversarial Networks](https://mlsanity.com/paper/1701.04862/) (2017)
- [Formal Limitations on the Measurement of Mutual Information](https://mlsanity.com/paper/1811.04251/) (2018)
- [Understanding the Limitations of Variational Mutual Information Estimators](https://mlsanity.com/paper/1910.06222/) (2019)
- [Non-Negative Bregman Divergence Minimization for Deep Direct Density Ratio Estimation](https://mlsanity.com/paper/2006.06979/) (2020)
- [Generative Adversarial Nets from a Density Ratio Estimation Perspective](https://mlsanity.com/paper/1610.02920/) (2016)
- [How Well Can Generative Adversarial Networks Learn Densities: A Nonparametric View](https://mlsanity.com/paper/1712.08244/) (2017)
- [Rates of convergence for density estimation with generative adversarial networks](https://mlsanity.com/paper/2102.00199/) (2021)
- [Density Ratio Estimation via Sampling along Generalized Geodesics on Statistical Manifolds](https://mlsanity.com/paper/2406.18806/) (2024)
- [Alpha-divergence loss function for neural density ratio estimation](https://mlsanity.com/paper/2402.02041/) (2024)

## Built on by (most cited first, 2 of 2)

- [Diffusion Secant Alignment for Score-Based Density Ratio Estimation](https://mlsanity.com/paper/2509.04852/) (2025)
- [The Capabilities and Limitations of Weak-to-Strong Generalization: Generalization and Calibration](https://mlsanity.com/paper/2502.01458/) (2025)

## Abstract (arXiv)

Density ratio estimation (DRE) is a core technique in machine learning used to capture relationships between two probability distributions. f-divergence loss functions, which are derived from variational representations of f-divergence, have become a standard choice in DRE for achieving cutting-edge performance. This study provides novel theoretical insights into DRE by deriving upper and lower bounds on the Lp errors through f-divergence loss functions. These bounds apply to any estimator belonging to a class of Lipschitz continuous estimators, irrespective of the specific f-divergence loss function employed. The derived bounds are expressed as a product involving the data dimensionality and the expected value of the density ratio raised to the p-th power. Notably, the lower bound includes an exponential term that depends on the Kullback--Leibler (KL) divergence, revealing that the Lp error increases significantly as the KL divergence grows when p > 1. This increase becomes even more pronounced as the value of p grows. The theoretical insights are validated through numerical experiments.