# Learning density ratios in causal inference using Bregman-Riesz regression

arXiv: https://arxiv.org/abs/2510.16127 · HTML: https://mlsanity.com/paper/2510.16127/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 38 of its 96 references that are in the map.
- First posted on arXiv 17 Oct 2025.

## Builds on (most cited first, 30 of 38)

- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution](https://mlsanity.com/paper/2310.16834/) (2023)
- [MINE: Mutual Information Neural Estimation](https://mlsanity.com/paper/1801.04062/) (2018)
- [Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning](https://mlsanity.com/paper/1706.03461/) (2017)
- [Direct and Indirect Effects](https://mlsanity.com/paper/1301.2300/) (2013)
- [Concrete Score Matching: Generalized Score Matching for Discrete Data](https://mlsanity.com/paper/2211.00802/) (2022)
- [Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processes](https://mlsanity.com/paper/1908.08526/) (2019)
- [Telescoping Density-Ratio Estimation](https://mlsanity.com/paper/2006.12204/) (2020)
- [Minimax Estimation of Conditional Moment Models](https://mlsanity.com/paper/2006.07201/) (2020)
- [Nonparametric methods for doubly robust estimation of continuous treatment effects](https://mlsanity.com/paper/1507.00747/) (2015)
- [Density Ratio Estimation via Infinitesimal Classification](https://mlsanity.com/paper/2111.11010/) (2021)
- [Approximate Residual Balancing: De-Biased Inference of Average Treatment Effects in High Dimensions](https://mlsanity.com/paper/1604.07125/) (2016)
- [Adversarial Estimation of Riesz Representers](https://mlsanity.com/paper/2101.00009/) (2020)
- [Covariate Balancing Propensity Score by Tailored Loss Functions](https://mlsanity.com/paper/1601.05890/) (2016)
- [Augmented Minimax Linear Estimation](https://mlsanity.com/paper/1712.00038/) (2017)
- [Featurized Density Ratio Estimation](https://mlsanity.com/paper/2107.02212/) (2021)
- [Nonparametric causal effects based on incremental propensity score interventions](https://mlsanity.com/paper/1704.00211/) (2017)
- [The Balancing Act in Causal Inference](https://mlsanity.com/paper/2110.14831/) (2021)
- [High-Dimensional Density Ratio Estimation with Extensions to Approximate Likelihood Computation](https://mlsanity.com/paper/1404.7063/) (2014)
- [Minimal Dispersion Approximately Balancing Weights: Asymptotic Properties and Practical Considerations](https://mlsanity.com/paper/1705.00998/) (2017)
- [Combining T-learning and DR-learning: a framework for oracle-efficient estimation of causal contrasts](https://mlsanity.com/paper/2402.01972/) (2024)
- [Permutation Weighting](https://mlsanity.com/paper/1901.01230/) (2019)
- [Non-parametric causal effects based on longitudinal modified treatment policies](https://mlsanity.com/paper/2006.01366/) (2020)
- [Density Ratio Estimation with Conditional Probability Paths](https://mlsanity.com/paper/2502.02300/) (2025)
- [Balanced off-policy evaluation in general action spaces](https://mlsanity.com/paper/1906.03694/) (2019)
- [On regularized Radon-Nikodym differentiation](https://mlsanity.com/paper/2308.07887/) (2023)
- [Optimal convex M-estimation via score matching](https://mlsanity.com/paper/2403.16688/) (2024)
- [Doubly robust inference via calibration](https://mlsanity.com/paper/2411.02771/) (2024)
- [DoubleGen: Debiased Generative Modeling of Counterfactuals](https://mlsanity.com/paper/2509.16842/) (2025)
- [Your copula is a classifier in disguise: classification-based copula density estimation](https://mlsanity.com/paper/2411.03014/) (2024)

## Abstract (arXiv)

The ratio of two probability density functions is a fundamental quantity that appears in many areas of statistics and machine learning, including causal inference, reinforcement learning, covariate shift, outlier detection, independence testing, importance sampling, and diffusion modeling. Naively estimating the numerator and denominator densities separately using, e.g., kernel density estimators, can lead to unstable performance and suffer from the curse of dimensionality as the number of covariates increases. For this reason, several methods have been developed for estimating the density ratio directly based on (a) Bregman divergences or (b) recasting the density ratio as the odds in a probabilistic classification model that predicts whether an observation is sampled from the numerator or denominator distribution. Additionally, the density ratio can be viewed as the Riesz representer of a continuous linear map, making it amenable to estimation via (c) minimization of the so-called Riesz loss, which was developed to learn the Riesz representer in the Riesz regression procedure in causal inference. In this paper we show that all three of these methods can be unified in a common framework, which we call Bregman--Riesz regression. We further show how data augmentation techniques can be used to apply density ratio learning methods to causal problems, where the numerator distribution typically represents an unobserved intervention. We show through simulations how the choice of Bregman divergence and data augmentation strategy can affect the performance of the resulting density ratio learner. A Python package is provided for researchers to apply Bregman--Riesz regression in practice using gradient boosting, neural networks, and kernel methods.