mlsanityData to 7 Oct 2026

Squared Neural Families: A New Class of Tractable Density Models

arXiv 2305.13552 ↗First posted 22 May 2023cs.LG cs.AI stat.MLcited by 12

Not on the map. Why?

Abstract (arXiv)

Flexible models for probability distributions are an essential ingredient in many machine learning tasks. We develop and investigate a new class of probability distributions, which we call a Squared Neural Family (SNEFY), formed by squaring the 2-norm of a neural network and normalising it with respect to a base measure. Following the reasoning similar to the well established connections between infinitely wide neural networks and Gaussian processes, we show that SNEFYs admit closed form normalising constants in many cases of interest, thereby resulting in flexible yet fully tractable density models. SNEFYs strictly generalise classical exponential families, are closed under conditioning, and have tractable marginal distributions. Their utility is illustrated on a variety of density estimation, conditional density estimation, and density estimation with missing data tasks.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 28 of its 54 references that are in the map. First posted on arXiv 22 May 2023.

Builds on

28 of 54 references.

References · 28

  1. 2013Auto-Encoding Variational Bayescited by 3,980
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  3. 2015Variational Inference with Normalizing Flowscited by 2,161
  4. 2016Density estimation using Real NVPcited by 2,029
  5. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
  6. 2017Deep Neural Networks as Gaussian Processescited by 617
  7. 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 329
  8. 2019Learning Likelihoods with Conditional Normalizing Flowscited by 170
  9. 2018Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processescited by 164
  10. 2020Neural Networks Fail to Learn Periodic Functions and How to Fix Itcited by 97
  11. 2023normflows: A PyTorch Package for Normalizing Flowscited by 56
  12. 2021Resampling Base Distributions of Normalizing Flowscited by 42
16 more
  1. 2020Non-parametric Models for Non-negative Functionscited by 40
  2. 2020MCFlow: Monte Carlo Flow Models for Data Imputationcited by 37
  3. 2019Autoregressive Energy Machinescited by 23
  4. 2019Neural Density Estimation and Likelihood-free Inferencecited by 23
  5. 2019Expressive Priors in Bayesian Neural Networks: Kernel Combinations and Periodic Functionscited by 22
  6. 2021PSD Representations for Effective Probability Modelscited by 22
  7. 2021Periodic Activation Functions Induce Stationaritycited by 21
  8. 2022Fast Neural Kernel Embeddings for General Activationscited by 18
  9. 2016Poisson intensity estimation with reproducing kernelscited by 14
  10. 2020Stationary Activations for Uncertainty Calibration in Deep Learningcited by 14
  11. 2017Invariance of Weight Distributions in Rectified MLPscited by 12
  12. 2017Fast Bayesian Intensity Estimation for the Permanental Processcited by 10
  13. 2019Richer priors for infinitely wide multi-layer perceptronscited by 9
  14. 2021Sampling from Arbitrary Functions via PSD Modelscited by 8
  15. 2020Avoiding Kernel Fixed Points: Computing with ELU and GELU Infinite Networkscited by 6
  16. 2021Training Deep Normalizing Flow Models in Highly Incomplete Data Scenarios with Prior Regularizationcited by 1

Built on by

Cited by year

Citing papers per year: 2023: 5; 2024: 4; 2025: 3.

12 papers.

  1. 2023Subtractive Mixture Models via Squaring: Representation and Learningcited by 39
  2. 2024Sum of Squares Circuitscited by 21
  3. 2024On the Relationship Between Monotone and Squared Probabilistic Circuitscited by 14
  4. 2023Probabilistic Integral Circuitscited by 8
  5. 2024Exact, Fast and Expressive Poisson Point Processes via Squared Neural Familiescited by 6
  6. 2025How to Square Tensor Networks and Circuits Without Squaring Themcited by 5
  7. 2023Sobolev Space Regularised Pre Density Modelscited by 2
  8. 2025Squared families: Searching beyond regular probability modelscited by 1
  9. 2025ActiveCQ: Active Estimation of Causal Quantitiescited by 1
  10. 2023Kernelised Normalising Flowscited by 0
  11. 20232-Cats: 2D Copula Approximating Transformscited by 0
  12. 2024Label Distribution Learning using the Squared Neural Family on the Probability Simplexcited by 0