# Telescoping Density-Ratio Estimation

arXiv: https://arxiv.org/abs/2006.12204 · HTML: https://mlsanity.com/paper/2006.12204/

## Key facts

- Cited by 108 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jun 2020.

## Built on by (most cited first, 30 of 108)

- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [Data Selection for Language Models via Importance Resampling](https://mlsanity.com/paper/2302.03169/) (2023)
- [Refining Generative Process with Discriminator Guidance in Score-based Diffusion Models](https://mlsanity.com/paper/2211.17091/) (2022)
- [Action Matching: Learning Stochastic Dynamics from Samples](https://mlsanity.com/paper/2210.06662/) (2022)
- [Density Ratio Estimation via Infinitesimal Classification](https://mlsanity.com/paper/2111.11010/) (2021)
- [Mandoline: Model Evaluation under Distribution Shift](https://mlsanity.com/paper/2107.00643/) (2021)
- [Compositional Score Modeling for Simulation-based Inference](https://mlsanity.com/paper/2209.14249/) (2022)
- [Featurized Density Ratio Estimation](https://mlsanity.com/paper/2107.02212/) (2021)
- [MINDE: Mutual Information Neural Diffusion Estimation](https://mlsanity.com/paper/2310.09031/) (2023)
- [Decomposed Mutual Information Estimation for Contrastive Representation Learning](https://mlsanity.com/paper/2106.13401/) (2021)
- [Training Unbiased Diffusion Models From Biased Dataset](https://mlsanity.com/paper/2403.01189/) (2024)
- [Beyond Statistical Similarity: Rethinking Metrics for Deep Generative Models in Engineering Design](https://mlsanity.com/paper/2302.02913/) (2023)
- [Mutual Information Estimation via Normalizing Flows](https://mlsanity.com/paper/2403.02187/) (2024)
- [Adaptive Multi-stage Density Ratio Estimation for Learning Latent Space Energy-based Model](https://mlsanity.com/paper/2209.08739/) (2022)
- [Estimating the Density Ratio between Distributions with High Discrepancy using Multinomial Logistic Regression](https://mlsanity.com/paper/2305.00869/) (2023)
- [Analyzing and Improving the Optimization Landscape of Noise-Contrastive Estimation](https://mlsanity.com/paper/2110.11271/) (2021)
- [A Short Survey on Importance Weighting for Machine Learning](https://mlsanity.com/paper/2403.10175/) (2024)
- [The Data Addition Dilemma](https://mlsanity.com/paper/2408.04154/) (2024)
- [Energy-Based Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2202.04933/) (2022)
- [Graphical vs. Deep Generative Models: Measuring the Impact of Differentially Private Mechanisms and Budgets on Utility](https://mlsanity.com/paper/2305.10994/) (2023)
- [Density Ratio Estimation with Conditional Probability Paths](https://mlsanity.com/paper/2502.02300/) (2025)
- [Constraining Generative Models for Engineering Design with Negative Data](https://mlsanity.com/paper/2306.15166/) (2023)
- [Diffusion Rejection Sampling](https://mlsanity.com/paper/2405.17880/) (2024)
- [Annealing Flow Generative Models Towards Sampling High-Dimensional and Multi-Modal Distributions](https://mlsanity.com/paper/2409.20547/) (2024)
- [Dequantified Diffusion-Schr"odinger Bridge for Density Ratio Estimation](https://mlsanity.com/paper/2505.05034/) (2025)
- [A Computationally Efficient Method for Learning Exponential Family Distributions](https://mlsanity.com/paper/2110.15397/) (2021)
- [Transfer Learning for Diffusion Models](https://mlsanity.com/paper/2405.16876/) (2024)
- [Data Retrieval with Importance Weights for Few-Shot Imitation Learning](https://mlsanity.com/paper/2509.01657/) (2025)
- [Direct Bias-Correction Term Estimation for Average Treatment Effect Estimation](https://mlsanity.com/paper/2509.22122/) (2025)
- [Computing high-dimensional optimal transport by flow neural networks](https://mlsanity.com/paper/2305.11857/) (2023)

## Abstract (arXiv)

Density-ratio estimation via classification is a cornerstone of unsupervised learning. It has provided the foundation for state-of-the-art methods in representation learning and generative modelling, with the number of use-cases continuing to proliferate. However, it suffers from a critical limitation: it fails to accurately estimate ratios p/q for which the two densities differ significantly. Empirically, we find this occurs whenever the KL divergence between p and q exceeds tens of nats. To resolve this limitation, we introduce a new framework, telescoping density-ratio estimation (TRE), that enables the estimation of ratios between highly dissimilar densities in high-dimensional spaces. Our experiments demonstrate that TRE can yield substantial improvements over existing single-ratio methods for mutual information estimation, representation learning and energy-based modelling.