mlsanityData to 14 Oct 2026

Overcoming Saturation in Density Ratio Estimation by Iterated Regularization

arXiv 2402.13891 ↗First posted 21 Feb 2024cs.LG stat.MLcited by 5

Not on the map. Why?

Abstract (arXiv)

Estimating the ratio of two probability densities from finitely many samples, is a central task in machine learning and statistics. In this work, we show that a large class of kernel methods for density ratio estimation suffers from error saturation, which prevents algorithms from achieving fast error convergence rates on highly regular learning problems. To resolve saturation, we introduce iterated regularization in density ratio estimation to achieve fast error rates. Our methods outperform its non-iteratively regularized versions on benchmarks for density ratio estimation as well as on large-scale evaluations for importance-weighted ensembling of deep unsupervised domain adaptation models.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 35 of its 98 references that are in the map. First posted on arXiv 21 Feb 2024.

Builds on

35 of 98 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  3. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  4. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,331
  5. 2014Deep Domain Confusion: Maximizing for Domain Invariancecited by 600
  6. 2017Positive-Unlabeled Learning with Non-Negative Risk Estimatorcited by 219
  7. 2018A DIRT-T Approach to Unsupervised Domain Adaptationcited by 215
  8. 2017Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learningcited by 192
  9. 2016Generalization Properties of Learning with Random Featurescited by 164
  10. 2016Correlation Alignment for Unsupervised Domain Adaptationcited by 147
  11. 2016Learning in Implicit Generative Modelscited by 125
  12. 2019Implicit Regularization of Discrete Gradient Dynamics in Linear Neural Networkscited by 124
23 more
  1. 2021Deep Subdomain Adaptation Network for Image Classificationcited by 119
  2. 2020Telescoping Density-Ratio Estimationcited by 108
  3. 2022Refining Generative Process with Discriminator Guidance in Score-based Diffusion Modelscited by 100
  4. 2019HoMM: Higher-order Moment Matching for Unsupervised Domain Adaptationcited by 89
  5. 2020Multi-Source Deep Domain Adaptation with Weak Supervision for Time-Series Sensor Datacited by 82
  6. 2016Optimal Rates For Regularization Of Statistical Inverse Learning Problemscited by 68
  7. 2018Optimal Rates for Spectral Algorithms with Least-Squares Regression over Hilbert Spacescited by 59
  8. 2020Non-Negative Bregman Divergence Minimization for Deep Direct Density Ratio Estimationcited by 45
  9. 2022ADATIME: A Benchmarking Suite for Domain Adaptation on Time Series Datacited by 45
  10. 2020A General Framework for Consistent Structured Prediction with Implicit Loss Embeddingscited by 35
  11. 2019On Minimum Discrepancy Estimation for Deep Domain Adaptationcited by 34
  12. 2021Featurized Density Ratio Estimationcited by 33
  13. 2018Finite-sample analysis of M-estimators using self-concordancecited by 30
  14. 2019Beyond Least-Squares: Fast Rates for Regularized Empirical Risk Minimization through Self-Concordancecited by 28
  15. 2019Globally Convergent Newton Methods for Ill-conditioned Generalized Self-concordant Lossescited by 17
  16. 2023Estimating the Density Ratio between Distributions with High Discrepancy using Multinomial Logistic Regressioncited by 17
  17. 2023Addressing Parameter Choice Issues in Unsupervised Domain Adaptation by Aggregationcited by 15
  18. 2019Kernel Conditional Density Operatorscited by 14
  19. 2023On regularized Radon-Nikodym differentiationcited by 10
  20. 2013Inverse Density as an Inverse Problem: The Fredholm Equation Approachcited by 9
  21. 2023Adaptive learning of density ratios in RKHScited by 8
  22. 2022A nonlinear conjugate gradient method with complexity guarantees and its application to nonconvex regressioncited by 4
  23. 2021Beyond Tikhonov: Faster Learning with Self-Concordant Losses via Iterative Regularizationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 2; 2026: 1.

5 papers.

  1. 2025Projection Pursuit Density Ratio Estimationcited by 5
  2. 2025Diffusion Secant Alignment for Score-Based Density Ratio Estimationcited by 3
  3. 2024Binary Losses for Density Ratio Estimationcited by 2
  4. 2026One-Step Score-Based Density Ratio Estimationcited by 1
  5. 2024Mutual Information Multinomial Estimationcited by 0