# Adaptive learning of density ratios in RKHS

arXiv: https://arxiv.org/abs/2307.16164 · HTML: https://mlsanity.com/paper/2307.16164/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 52 references that are in the map.
- First posted on arXiv 30 Jul 2023.

## Builds on (most cited first, 9 of 9)

- [Generalization Properties of Learning with Random Features](https://mlsanity.com/paper/1602.04474/) (2016)
- [Learning in Implicit Generative Models](https://mlsanity.com/paper/1610.03483/) (2016)
- [Optimal Rates For Regularization Of Statistical Inverse Learning Problems](https://mlsanity.com/paper/1604.04054/) (2016)
- [Beyond Least-Squares: Fast Rates for Regularized Empirical Risk Minimization through Self-Concordance](https://mlsanity.com/paper/1902.03046/) (2019)
- [Addressing Parameter Choice Issues in Unsupervised Domain Adaptation by Aggregation](https://mlsanity.com/paper/2305.01281/) (2023)
- [Kernel Conditional Density Operators](https://mlsanity.com/paper/1905.11255/) (2019)
- [Capacity dependent analysis for functional online learning algorithms](https://mlsanity.com/paper/2209.12198/) (2022)
- [Inverse Density as an Inverse Problem: The Fredholm Equation Approach](https://mlsanity.com/paper/1304.5575/) (2013)
- [Adaptivity for Regularized Kernel Methods by Lepskii's Principle](https://mlsanity.com/paper/1804.05433/) (2018)

## Built on by (most cited first, 8 of 8)

- [Overcoming Saturation in Density Ratio Estimation by Iterated Regularization](https://mlsanity.com/paper/2402.13891/) (2024)
- [Spectral Algorithms under Covariate Shift](https://mlsanity.com/paper/2504.12625/) (2025)
- [Diffusion Secant Alignment for Score-Based Density Ratio Estimation](https://mlsanity.com/paper/2509.04852/) (2025)
- [Binary Losses for Density Ratio Estimation](https://mlsanity.com/paper/2407.01371/) (2024)
- [Kernel Density Machines](https://mlsanity.com/paper/2504.21419/) (2025)
- [One-Step Score-Based Density Ratio Estimation](https://mlsanity.com/paper/2604.10672/) (2026)
- [Target-Aware Linear Regression Under Distribution Shift](https://mlsanity.com/paper/2606.22775/) (2026)
- [Local polynomial density ratio estimation](https://mlsanity.com/paper/2609.38412/) (2026)

## Abstract (arXiv)

Estimating the ratio of two probability densities from finitely many observations of the densities is a central problem in machine learning and statistics with applications in two-sample testing, divergence estimation, generative modeling, covariate shift adaptation, conditional density estimation, and novelty detection. In this work, we analyze a large class of density ratio estimation methods that minimize a regularized Bregman divergence between the true density ratio and a model in a reproducing kernel Hilbert space (RKHS). We derive new finite-sample error bounds, and we propose a Lepskii type parameter choice principle that minimizes the bounds without knowledge of the regularity of the density ratio. In the special case of quadratic loss, our method adaptively achieves a minimax optimal error rate. A numerical illustration is provided.