# PSD Representations for Effective Probability Models

arXiv: https://arxiv.org/abs/2106.16116 · HTML: https://mlsanity.com/paper/2106.16116/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 44 references that are in the map.
- First posted on arXiv 30 Jun 2021.

## Builds on (most cited first, 10 of 10)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Kernel Mean Embedding of Distributions: A Review and Beyond](https://mlsanity.com/paper/1605.09522/) (2016)
- [Generalization Properties of Learning with Random Features](https://mlsanity.com/paper/1602.04474/) (2016)
- [Density Estimation in Infinite Dimensional Exponential Families](https://mlsanity.com/paper/1312.3516/) (2013)
- [Massively scalable Sinkhorn distances via the Nystr"om method](https://mlsanity.com/paper/1812.05189/) (2018)
- [Non-parametric Models for Non-negative Functions](https://mlsanity.com/paper/2007.03926/) (2020)
- [A General Framework for Consistent Structured Prediction with Implicit Loss Embeddings](https://mlsanity.com/paper/2002.05424/) (2020)
- [Finding Global Minima via Kernel Approximations](https://mlsanity.com/paper/2012.11978/) (2020)
- [Interpolation and Learning with Scale Dependent Kernels](https://mlsanity.com/paper/2006.09984/) (2020)
- [A Statistical Perspective on Coreset Density Estimation](https://mlsanity.com/paper/2011.04907/) (2020)

## Built on by (most cited first, 23 of 23)

- [Subtractive Mixture Models via Squaring: Representation and Learning](https://mlsanity.com/paper/2310.00724/) (2023)
- [Sum of Squares Circuits](https://mlsanity.com/paper/2408.11778/) (2024)
- [On the Relationship Between Monotone and Squared Probabilistic Circuits](https://mlsanity.com/paper/2408.00876/) (2024)
- [Squared Neural Families: A New Class of Tractable Density Models](https://mlsanity.com/paper/2305.13552/) (2023)
- [Sampling from Arbitrary Functions via PSD Models](https://mlsanity.com/paper/2110.10527/) (2021)
- [Exact, Fast and Expressive Poisson Point Processes via Squared Neural Families](https://mlsanity.com/paper/2402.09608/) (2024)
- [Learning PSD-valued functions using kernel sums-of-squares](https://mlsanity.com/paper/2111.11306/) (2021)
- [How to Square Tensor Networks and Circuits Without Squaring Them](https://mlsanity.com/paper/2512.17090/) (2025)
- [Non-Parametric Learning of Stochastic Differential Equations with Non-asymptotic Fast Rates of Convergence](https://mlsanity.com/paper/2305.15557/) (2023)
- [Non-Convex Optimization with Certificates and Fast Rates Through Kernel Sums of Squares](https://mlsanity.com/paper/2204.04970/) (2022)
- [Efficient Sampling of Stochastic Differential Equations with Positive Semi-Definite Models](https://mlsanity.com/paper/2303.17109/) (2023)
- [Squared families: Searching beyond regular probability models](https://mlsanity.com/paper/2503.21128/) (2025)
- [Scalable and adaptive prediction bands with kernel sum-of-squares](https://mlsanity.com/paper/2505.21039/) (2025)
- [How to Approximate Inference with Subtractive Mixture Models](https://mlsanity.com/paper/2604.16714/) (2026)
- [Measuring dissimilarity with diffeomorphism invariance](https://mlsanity.com/paper/2202.05614/) (2022)
- [GloptiNets: Scalable Non-Convex Optimization with Certificates](https://mlsanity.com/paper/2306.14932/) (2023)
- [Kernelised Normalising Flows](https://mlsanity.com/paper/2307.14839/) (2023)
- [2-Cats: 2D Copula Approximating Transforms](https://mlsanity.com/paper/2309.16391/) (2023)
- [Closed-form Filtering for Non-linear Systems](https://mlsanity.com/paper/2402.09796/) (2024)
- [Scalable Expectation Estimation with Subtractive Mixture Models](https://mlsanity.com/paper/2503.21346/) (2025)
- [Learning Markov Processes as Sum-of-Square Forms for Analytical Belief Propagation](https://mlsanity.com/paper/2604.07525/) (2026)
- [Distributionally Faithful Imputation via Positive Semi-Definite Kernel Density Estimation](https://mlsanity.com/paper/2607.07767/) (2026)
- [Density-Matrix Reinforcement Learning](https://mlsanity.com/paper/2610.17214/) (2026)

## Abstract (arXiv)

Finding a good way to model probability densities is key to probabilistic inference. An ideal model should be able to concisely approximate any probability while being also compatible with two main operations: multiplications of two models (product rule) and marginalization with respect to a subset of the random variables (sum rule). In this work, we show that a recently proposed class of positive semi-definite (PSD) models for non-negative functions is particularly suited to this end. In particular, we characterize both approximation and generalization capabilities of PSD models, showing that they enjoy strong theoretical guarantees. Moreover, we show that we can perform efficiently both sum and product rule in closed form via matrix operations, enjoying the same versatility of mixture models. Our results open the way to applications of PSD models to density estimation, decision theory and inference.