# Scalable Expectation Estimation with Subtractive Mixture Models

arXiv: https://arxiv.org/abs/2503.21346 · HTML: https://mlsanity.com/paper/2503.21346/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 37 references that are in the map.
- First posted on arXiv 27 Mar 2025.

## Builds on (most cited first, 17 of 17)

- [Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC](https://mlsanity.com/paper/1507.04544/) (2015)
- [Non-parametric Models for Non-negative Functions](https://mlsanity.com/paper/2007.03926/) (2020)
- [On the Latent Variable Interpretation in Sum-Product Networks](https://mlsanity.com/paper/1601.06180/) (2016)
- [Subtractive Mixture Models via Squaring: Representation and Learning](https://mlsanity.com/paper/2310.00724/) (2023)
- [Assessing Fairness in the Presence of Missing Data](https://mlsanity.com/paper/2112.04899/) (2021)
- [PSD Representations for Effective Probability Models](https://mlsanity.com/paper/2106.16116/) (2021)
- [What is the Relationship between Tensor Factorizations and Circuits (and How Can We Exploit it)?](https://mlsanity.com/paper/2409.07953/) (2024)
- [Sum of Squares Circuits](https://mlsanity.com/paper/2408.11778/) (2024)
- [Safe Adaptive Importance Sampling](https://mlsanity.com/paper/1711.02637/) (2017)
- [Variational Mixture of Normalizing Flows](https://mlsanity.com/paper/2009.00585/) (2020)
- [Adaptive Importance Sampling meets Mirror Descent: a Bias-variance tradeoff](https://mlsanity.com/paper/2110.15590/) (2021)
- [EigenVI: score-based variational inference with orthogonal function expansions](https://mlsanity.com/paper/2410.24054/) (2024)
- [Adaptive normalization for IPW estimation](https://mlsanity.com/paper/2106.07695/) (2021)
- [Learning Negative Mixture Models by Tensor Decompositions](https://mlsanity.com/paper/1403.4224/) (2014)
- [Efficient Mixture Learning in Black-Box Variational Inference](https://mlsanity.com/paper/2406.07083/) (2024)
- [Simulating signed mixtures](https://mlsanity.com/paper/2401.16828/) (2024)
- [Generalizing self-normalized importance sampling with couplings](https://mlsanity.com/paper/2406.19974/) (2024)

## Abstract (arXiv)

Many Monte Carlo (MC) and importance sampling (IS) methods use mixture models (MMs) for their simplicity and ability to capture multimodal distributions. Recently, subtractive mixture models (SMMs), i.e. MMs with negative coefficients, have shown greater expressiveness and success in generative modeling. However, their negative parameters complicate sampling, requiring costly auto-regressive techniques or accept-reject algorithms that do not scale in high dimensions. In this work, we use the difference representation of SMMs to construct an unbiased IS estimator (DeltaEx) that removes the need to sample from the SMM, enabling high-dimensional expectation estimation with SMMs. In our experiments, we show that DeltaEx can achieve comparable estimation quality to auto-regressive sampling while being considerably faster in MC estimation. Moreover, we conduct initial experiments with DeltaEx using hand-crafted proposals, gaining first insights into how to construct safe proposals for DeltaEx.