# Squared Neural Families: A New Class of Tractable Density Models

arXiv: https://arxiv.org/abs/2305.13552 · HTML: https://mlsanity.com/paper/2305.13552/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 54 references that are in the map.
- First posted on arXiv 22 May 2023.

## Builds on (most cited first, 28 of 28)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [Breaking the Curse of Dimensionality with Convex Neural Networks](https://mlsanity.com/paper/1412.8690/) (2014)
- [Learning Likelihoods with Conditional Normalizing Flows](https://mlsanity.com/paper/1912.00042/) (2019)
- [Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes](https://mlsanity.com/paper/1810.05148/) (2018)
- [Neural Networks Fail to Learn Periodic Functions and How to Fix It](https://mlsanity.com/paper/2006.08195/) (2020)
- [normflows: A PyTorch Package for Normalizing Flows](https://mlsanity.com/paper/2302.12014/) (2023)
- [Resampling Base Distributions of Normalizing Flows](https://mlsanity.com/paper/2110.15828/) (2021)
- [Non-parametric Models for Non-negative Functions](https://mlsanity.com/paper/2007.03926/) (2020)
- [MCFlow: Monte Carlo Flow Models for Data Imputation](https://mlsanity.com/paper/2003.12628/) (2020)
- [Autoregressive Energy Machines](https://mlsanity.com/paper/1904.05626/) (2019)
- [Neural Density Estimation and Likelihood-free Inference](https://mlsanity.com/paper/1910.13233/) (2019)
- [PSD Representations for Effective Probability Models](https://mlsanity.com/paper/2106.16116/) (2021)
- [Expressive Priors in Bayesian Neural Networks: Kernel Combinations and Periodic Functions](https://mlsanity.com/paper/1905.06076/) (2019)
- [Periodic Activation Functions Induce Stationarity](https://mlsanity.com/paper/2110.13572/) (2021)
- [Fast Neural Kernel Embeddings for General Activations](https://mlsanity.com/paper/2209.04121/) (2022)
- [Poisson intensity estimation with reproducing kernels](https://mlsanity.com/paper/1610.08623/) (2016)
- [Stationary Activations for Uncertainty Calibration in Deep Learning](https://mlsanity.com/paper/2010.09494/) (2020)
- [Invariance of Weight Distributions in Rectified MLPs](https://mlsanity.com/paper/1711.09090/) (2017)
- [Fast Bayesian Intensity Estimation for the Permanental Process](https://mlsanity.com/paper/1701.03535/) (2017)
- [Richer priors for infinitely wide multi-layer perceptrons](https://mlsanity.com/paper/1911.12927/) (2019)
- [Sampling from Arbitrary Functions via PSD Models](https://mlsanity.com/paper/2110.10527/) (2021)
- [Avoiding Kernel Fixed Points: Computing with ELU and GELU Infinite Networks](https://mlsanity.com/paper/2002.08517/) (2020)
- [Training Deep Normalizing Flow Models in Highly Incomplete Data Scenarios with Prior Regularization](https://mlsanity.com/paper/2104.01482/) (2021)

## Built on by (most cited first, 12 of 12)

- [Subtractive Mixture Models via Squaring: Representation and Learning](https://mlsanity.com/paper/2310.00724/) (2023)
- [Sum of Squares Circuits](https://mlsanity.com/paper/2408.11778/) (2024)
- [On the Relationship Between Monotone and Squared Probabilistic Circuits](https://mlsanity.com/paper/2408.00876/) (2024)
- [Probabilistic Integral Circuits](https://mlsanity.com/paper/2310.16986/) (2023)
- [Exact, Fast and Expressive Poisson Point Processes via Squared Neural Families](https://mlsanity.com/paper/2402.09608/) (2024)
- [How to Square Tensor Networks and Circuits Without Squaring Them](https://mlsanity.com/paper/2512.17090/) (2025)
- [Sobolev Space Regularised Pre Density Models](https://mlsanity.com/paper/2307.13763/) (2023)
- [Squared families: Searching beyond regular probability models](https://mlsanity.com/paper/2503.21128/) (2025)
- [ActiveCQ: Active Estimation of Causal Quantities](https://mlsanity.com/paper/2509.24293/) (2025)
- [Kernelised Normalising Flows](https://mlsanity.com/paper/2307.14839/) (2023)
- [2-Cats: 2D Copula Approximating Transforms](https://mlsanity.com/paper/2309.16391/) (2023)
- [Label Distribution Learning using the Squared Neural Family on the Probability Simplex](https://mlsanity.com/paper/2412.07324/) (2024)

## Abstract (arXiv)

Flexible models for probability distributions are an essential ingredient in many machine learning tasks. We develop and investigate a new class of probability distributions, which we call a Squared Neural Family (SNEFY), formed by squaring the 2-norm of a neural network and normalising it with respect to a base measure. Following the reasoning similar to the well established connections between infinitely wide neural networks and Gaussian processes, we show that SNEFYs admit closed form normalising constants in many cases of interest, thereby resulting in flexible yet fully tractable density models. SNEFYs strictly generalise classical exponential families, are closed under conditioning, and have tractable marginal distributions. Their utility is illustrated on a variety of density estimation, conditional density estimation, and density estimation with missing data tasks.