# Non-Negative Bregman Divergence Minimization for Deep Direct Density Ratio Estimation

arXiv: https://arxiv.org/abs/2006.06979 · HTML: https://mlsanity.com/paper/2006.06979/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2020.

## Built on by (most cited first, 30 of 45)

- [Refining Generative Process with Discriminator Guidance in Score-based Diffusion Models](https://mlsanity.com/paper/2211.17091/) (2022)
- [Estimating the Density Ratio between Distributions with High Discrepancy using Multinomial Logistic Regression](https://mlsanity.com/paper/2305.00869/) (2023)
- [Adapting to Continuous Covariate Shift via Online Density Ratio Estimation](https://mlsanity.com/paper/2302.02552/) (2023)
- [Deep Bregman Divergence for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2109.07455/) (2021)
- [Towards a Unified Analysis of Kernel-based Methods Under Covariate Shift](https://mlsanity.com/paper/2310.08237/) (2023)
- [Dequantified Diffusion-Schr"odinger Bridge for Density Ratio Estimation](https://mlsanity.com/paper/2505.05034/) (2025)
- [Direct Bias-Correction Term Estimation for Average Treatment Effect Estimation](https://mlsanity.com/paper/2509.22122/) (2025)
- [Utility Theory of Synthetic Data Generation](https://mlsanity.com/paper/2305.10015/) (2023)
- [Computing high-dimensional optimal transport by flow neural networks](https://mlsanity.com/paper/2305.11857/) (2023)
- [Positive-Unlabeled Classification under Class-Prior Shift: A Prior-invariant Approach Based on Density Ratio Estimation](https://mlsanity.com/paper/2107.05045/) (2021)
- [Learning Causal Models from Conditional Moment Restrictions by Importance Weighting](https://mlsanity.com/paper/2108.01312/) (2021)
- [Nearest Neighbor Matching as Least Squares Density Ratio Estimation and Riesz Regression](https://mlsanity.com/paper/2510.24433/) (2025)
- [Meta-Learning for Relative Density-Ratio Estimation](https://mlsanity.com/paper/2107.00801/) (2021)
- [Federated Learning under Covariate Shifts with Generalization Guarantees](https://mlsanity.com/paper/2306.05325/) (2023)
- [Double Debiased Covariate Shift Adaptation Robust to Density-Ratio Estimation](https://mlsanity.com/paper/2310.16638/) (2023)
- [Direct Density Ratio Optimization: A Statistically Consistent Approach to Aligning Large Language Models](https://mlsanity.com/paper/2505.07558/) (2025)
- [Learning from positive and unlabeled examples -Finite size sample bounds](https://mlsanity.com/paper/2507.07354/) (2025)
- [A Unified Framework for Multi-distribution Density Ratio Estimation](https://mlsanity.com/paper/2112.03440/) (2021)
- [Overcoming Saturation in Density Ratio Estimation by Iterated Regularization](https://mlsanity.com/paper/2402.13891/) (2024)
- [Projection Pursuit Density Ratio Estimation](https://mlsanity.com/paper/2506.00866/) (2025)
- [Direct Debiased Machine Learning via Bregman Divergence Minimization](https://mlsanity.com/paper/2510.23534/) (2025)
- [Approximate Data Deletion in Generative Models](https://mlsanity.com/paper/2206.14439/) (2022)
- [Riesz Regression As Direct Density Ratio Estimation](https://mlsanity.com/paper/2511.04568/) (2025)
- [A Minimum Variance Path Principle for Accurate and Stable Score-Based Density Ratio Estimation](https://mlsanity.com/paper/2602.00834/) (2026)
- [Estimation of Local Average Treatment Effect by Data Combination](https://mlsanity.com/paper/2109.05175/) (2021)
- [Asymptotically Unbiased Synthetic Control Methods by Moment Matching](https://mlsanity.com/paper/2307.11127/) (2023)
- [Meta-learning for Positive-unlabeled Classification](https://mlsanity.com/paper/2406.03680/) (2024)
- [Binary Losses for Density Ratio Estimation](https://mlsanity.com/paper/2407.01371/) (2024)
- [Bounds on Lp errors in density ratio estimation via f-divergence loss functions](https://mlsanity.com/paper/2410.01516/) (2024)
- [A Unified Theory for Causal Inference: Direct Debiased Machine Learning via Bregman-Riesz Regression](https://mlsanity.com/paper/2510.26783/) (2025)

## Abstract (arXiv)

Density ratio estimation (DRE) is at the core of various machine learning tasks such as anomaly detection and domain adaptation. In existing studies on DRE, methods based on Bregman divergence (BD) minimization have been extensively studied. However, BD minimization when applied with highly flexible models, such as deep neural networks, tends to suffer from what we call train-loss hacking, which is a source of overfitting caused by a typical characteristic of empirical BD estimators. In this paper, to mitigate train-loss hacking, we propose a non-negative correction for empirical BD estimators. Theoretically, we confirm the soundness of the proposed method through a generalization error bound. Through our experiments, the proposed methods show a favorable performance in inlier-based outlier detection.