mlsanityData to 7 Oct 2026

Featurized Density Ratio Estimation

arXiv 2107.02212 ↗First posted 5 Jul 2021cs.LG stat.MLcited by 33

Not on the map. Why?

Abstract (arXiv)

Density ratio estimation serves as an important technique in the unsupervised machine learning toolbox. However, such ratios are difficult to estimate for complex, high-dimensional data, particularly when the densities of interest are sufficiently different. In our work, we propose to leverage an invertible generative model to map the two distributions into a common feature space prior to estimation. This featurization brings the densities closer together in latent space, sidestepping pathological scenarios where the learned density ratios in input space can be arbitrarily inaccurate. At the same time, the invertibility of our feature map guarantees that the ratios computed in feature space are equivalent to those in input space. Empirically, we demonstrate the efficacy of our approach in a variety of downstream tasks that require access to accurate density ratios such as mutual information estimation, targeted sampling in deep generative models, and classification with data augmentation.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Builds on 30 of its 59 references that are in the map. First posted on arXiv 5 Jul 2021.

Builds on

30 of 59 references.

References · 30

  1. 2016Matching Networks for One Shot Learningcited by 2,246
  2. 2015Variational Inference with Normalizing Flowscited by 2,161
  3. 2016Density estimation using Real NVPcited by 2,029
  4. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  5. 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,323
  6. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
  7. 2017Masked Autoregressive Flow for Density Estimationcited by 680
  8. 2018MINE: Mutual Information Neural Estimationcited by 680
  9. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  10. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 584
  11. 2019On Variational Bounds of Mutual Informationcited by 550
  12. 2016f-GAN: Training Generative Neural Samplers using Variational Divergence Minimizationcited by 504
18 more
  1. 2018Invertible Residual Networkscited by 364
  2. 2015MADE: Masked Autoencoder for Distribution Estimationcited by 306
  3. 2017Data Augmentation Generative Adversarial Networkscited by 278
  4. 2019Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Designcited by 258
  5. 2019Learning Likelihoods with Conditional Normalizing Flowscited by 170
  6. 2019Understanding the Limitations of Variational Mutual Information Estimatorscited by 140
  7. 2018i-RevNet: Deep Invertible Networkscited by 137
  8. 2019Fair Generative Modeling via Weak Supervisioncited by 114
  9. 2020Telescoping Density-Ratio Estimationcited by 108
  10. 2016Likelihood-free inference by ratio estimationcited by 80
  11. 2016Auxiliary Deep Generative Modelscited by 72
  12. 2019Bias Correction of Learned Generative Models using Likelihood-Free Importance Weightingcited by 70
  13. 2019Hybrid Models with Deep and Invertible Featurescited by 41
  14. 2019MintNet: Building Invertible Neural Networks with Masked Convolutionscited by 35
  15. 2019A RAD approach to deep mixture modelscited by 18
  16. 2021Nearest Neighbor-based Importance Weightingcited by 9
  17. 2013Constructive Setting of the Density Ratio Estimation Problem and its Rigorous Solutioncited by 4
  18. 2017Bayesian Semisupervised Learning with Deep Generative Modelscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 3; 2023: 4; 2024: 10; 2025: 10; 2026: 4.

33 papers.

Sort:

By area:

  1. 2023Data Selection for Language Models via Importance Resamplingcited by 322
  2. 2021Density Ratio Estimation via Infinitesimal Classificationcited by 59
  3. 2023Estimating the Density Ratio between Distributions with High Discrepancy using Multinomial Logistic Regressioncited by 17
  4. 2024A Short Survey on Importance Weighting for Machine Learningcited by 15
  5. 2025Dequantified Diffusion-Schr"odinger Bridge for Density Ratio Estimationcited by 11
  6. 2024Incentives in Private Collaborative Machine Learningcited by 10
  7. 2025Preference Optimization by Estimating the Ratio of the Data Distributioncited by 9
  8. 2023Computing high-dimensional optimal transport by flow neural networkscited by 8
  9. 2024Density Ratio Estimation via Sampling along Generalized Geodesics on Statistical Manifoldscited by 8
  10. 2024Your copula is a classifier in disguise: classification-based copula density estimationcited by 8
  11. 2025Diffusion and Flow-based Copulas: Forgetting and Remembering Dependenciescited by 7
  12. 2021A Unified Framework for Multi-distribution Density Ratio Estimationcited by 5
21 more
  1. 2024Overcoming Saturation in Density Ratio Estimation by Iterated Regularizationcited by 5
  2. 2025Projection Pursuit Density Ratio Estimationcited by 5
  3. 2024Expected information gain estimation via density approximations: Sample allocation and dimension reductioncited by 4
  4. 2022Approximate Data Deletion in Generative Modelscited by 3
  5. 2024General Frameworks for Conditional Two-Sample Testingcited by 3
  6. 2025Diffusion Secant Alignment for Score-Based Density Ratio Estimationcited by 3
  7. 2026A Minimum Variance Path Principle for Accurate and Stable Score-Based Density Ratio Estimationcited by 3
  8. 2022Off-Policy Evaluation in Embedded Spacescited by 2
  9. 2023Density Ratio Estimation and Neyman Pearson Classification with Missing Datacited by 2
  10. 2024Binary Losses for Density Ratio Estimationcited by 2
  11. 2026On the Difficulty of Learning a Meta-network for Training Data Selectioncited by 2
  12. 2025Enhanced Importance Sampling through Latent Space Exploration in Normalizing Flowscited by 1
  13. 2025Two-sample comparison through additive tree models for density ratioscited by 1
  14. 2026One-Step Score-Based Density Ratio Estimationcited by 1
  15. 2022Unified Perspective on Probability Divergence via Maximum Likelihood Density Ratio Estimation: Bridging KL-Divergence and Integral Probability Metricscited by 0
  16. 2024Mutual Information Multinomial Estimationcited by 0
  17. 2024A density ratio framework for evaluating the utility of synthetic datacited by 0
  18. 2025Neural Total Variation Distance Estimators for Changepoint Detection in News Datacited by 0
  19. 2025Learning density ratios in causal inference using Bregman-Riesz regressioncited by 0
  20. 2025Robust and Sparse Estimation of Unbounded Density Ratio under Heavy Contaminationcited by 0
  21. 2026The geometry of invariant learning: an information-theoretic analysis of data augmentation and generalizationcited by 0