mlsanityData to 7 Oct 2026

Telescoping Density-Ratio Estimation

arXiv 2006.12204 ↗First posted 22 Jun 2020stat.ML cs.LGcited by 108

Not on the map. Why?

Abstract (arXiv)

Density-ratio estimation via classification is a cornerstone of unsupervised learning. It has provided the foundation for state-of-the-art methods in representation learning and generative modelling, with the number of use-cases continuing to proliferate. However, it suffers from a critical limitation: it fails to accurately estimate ratios p/q for which the two densities differ significantly. Empirically, we find this occurs whenever the KL divergence between p and q exceeds tens of nats. To resolve this limitation, we introduce a new framework, telescoping density-ratio estimation (TRE), that enables the estimation of ratios between highly dissimilar densities in high-dimensional spaces. Our experiments demonstrate that TRE can yield substantial improvements over existing single-ratio methods for mutual information estimation, representation learning and energy-based modelling.

Key facts. Cited by 108 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 12; 2023: 20; 2024: 22; 2025: 28; 2026: 14.

108 papers.

Sort:

By area:

  1. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  2. 2023Data Selection for Language Models via Importance Resamplingcited by 322
  3. 2022Refining Generative Process with Discriminator Guidance in Score-based Diffusion Modelscited by 98
  4. 2022Action Matching: Learning Stochastic Dynamics from Samplescited by 94
  5. 2021Density Ratio Estimation via Infinitesimal Classificationcited by 59
  6. 2021Mandoline: Model Evaluation under Distribution Shiftcited by 55
  7. 2022Compositional Score Modeling for Simulation-based Inferencecited by 46
  8. 2021Featurized Density Ratio Estimationcited by 33
  9. 2023MINDE: Mutual Information Neural Diffusion Estimationcited by 31
  10. 2024Training Unbiased Diffusion Models From Biased Datasetcited by 25
  11. 2021Decomposed Mutual Information Estimation for Contrastive Representation Learningcited by 24
  12. 2023Beyond Statistical Similarity: Rethinking Metrics for Deep Generative Models in Engineering Designcited by 22
38 more
  1. 2024Mutual Information Estimation via Normalizing Flowscited by 22
  2. 2022Adaptive Multi-stage Density Ratio Estimation for Learning Latent Space Energy-based Modelcited by 18
  3. 2023Estimating the Density Ratio between Distributions with High Discrepancy using Multinomial Logistic Regressioncited by 17
  4. 2021Analyzing and Improving the Optimization Landscape of Noise-Contrastive Estimationcited by 15
  5. 2024A Short Survey on Importance Weighting for Machine Learningcited by 15
  6. 2024The Data Addition Dilemmacited by 15
  7. 2022Energy-Based Contrastive Learning of Visual Representationscited by 13
  8. 2023Graphical vs. Deep Generative Models: Measuring the Impact of Differentially Private Mechanisms and Budgets on Utilitycited by 13
  9. 2025Density Ratio Estimation with Conditional Probability Pathscited by 13
  10. 2023Constraining Generative Models for Engineering Design with Negative Datacited by 11
  11. 2024Diffusion Rejection Samplingcited by 11
  12. 2024Annealing Flow Generative Models Towards Sampling High-Dimensional and Multi-Modal Distributionscited by 11
  13. 2025Dequantified Diffusion-Schr"odinger Bridge for Density Ratio Estimationcited by 11
  14. 2021A Computationally Efficient Method for Learning Exponential Family Distributionscited by 10
  15. 2024Transfer Learning for Diffusion Modelscited by 10
  16. 2025Data Retrieval with Importance Weights for Few-Shot Imitation Learningcited by 10
  17. 2023Computing high-dimensional optimal transport by flow neural networkscited by 8
  18. 2023Minimizing f-Divergences by Interpolating Velocity Fieldscited by 8
  19. 2024"Noisier" Noise Contrastive Eestimation is (Almost) Maximum Likelihoodcited by 8
  20. 2024Information Theoretic Text-to-Image Alignmentcited by 8
  21. 2024Density Ratio Estimation via Sampling along Generalized Geodesics on Statistical Manifoldscited by 8
  22. 2024Your copula is a classifier in disguise: classification-based copula density estimationcited by 8
  23. 2025Direct Bias-Correction Term Estimation for Average Treatment Effect Estimationcited by 8
  24. 2022A unified framework for dataset shift diagnosticscited by 7
  25. 2025InfoBridge: Mutual Information estimation via Bridge Matchingcited by 7
  26. 2025BoltzNCE: Learning Likelihoods for Boltzmann Generation with Stochastic Interpolants and Noise Contrastive Estimationcited by 7
  27. 2025Diffusion and Flow-based Copulas: Forgetting and Remembering Dependenciescited by 7
  28. 2021Meta-Learning for Relative Density-Ratio Estimationcited by 6
  29. 2022Pitfalls of Gaussians as a noise distribution in NCEcited by 6
  30. 2023On the Effectiveness of Hybrid Mutual Information Estimationcited by 6
  31. 2024Synthetic Tabular Data Validation: A Divergence-Based Approachcited by 6
  32. 2025Nearest Neighbor Matching as Least Squares Density Ratio Estimation and Riesz Regressioncited by 6
  33. 2021A Unified Framework for Multi-distribution Density Ratio Estimationcited by 5
  34. 2022Statistical applications of contrastive learningcited by 5
  35. 2023Learning Unnormalized Statistical Models via Compositional Optimizationcited by 5
  36. 2024Overcoming Saturation in Density Ratio Estimation by Iterated Regularizationcited by 5
  37. 2024Your Diffusion Model is Secretly a Noise Classifier and Benefits from Contrastive Trainingcited by 5
  38. 2025Projection Pursuit Density Ratio Estimationcited by 5

50 most cited of 108 All 108 (JSON)