mlsanityData to 7 Oct 2026

Subtractive Mixture Models via Squaring: Representation and Learning

arXiv 2310.00724 ↗First posted 1 Oct 2023cs.LG cs.AIcited by 39

Not on the map. Why?

Abstract (arXiv)

Mixture models are traditionally represented and learned by adding several distributions as components. Allowing mixtures to subtract probability mass or density can drastically reduce the number of components needed to model complex distributions. However, learning such subtractive mixtures while ensuring they still encode a non-negative function is challenging. We investigate how to learn and perform inference on deep subtractive mixtures by squaring them. We do this in the framework of probabilistic circuits, which enable us to represent tensorized mixtures and generalize several other subtractive models. We theoretically prove that the class of squared circuits allowing subtractions can be exponentially more expressive than traditional additive mixtures; and, we empirically show this increased expressiveness on a series of real-world distribution estimation tasks.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Builds on 31 of its 95 references that are in the map. First posted on arXiv 1 Oct 2023.

Builds on

31 of 95 references.

References · 31

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016Density estimation using Real NVPcited by 2,029
  3. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
  4. 2017Masked Autoregressive Flow for Density Estimationcited by 680
  5. 2019Neural Spline Flowscited by 590
  6. 2015MADE: Masked Autoencoder for Distribution Estimationcited by 306
  7. 2022Semantic Probabilistic Layers for Neuro-Symbolic Learningcited by 103
  8. 2020Einsum Networks: Fast and Scalable Learning of Tractable Probabilistic Circuitscited by 102
  9. 2018Differentiable Learning of Quantum Circuit Born Machinecited by 90
  10. 2017Unsupervised Generative Modeling Using Matrix Product Statescited by 87
  11. 2018Towards Quantum Machine Learning with Tensor Networkscited by 74
  12. 2023Tractable Control for Autoregressive Language Generationcited by 65
19 more
  1. 2019Tree Tensor Networks for Generative Modelingcited by 63
  2. 2019Expressive power of tensor-network factorizations for probabilistic modeling, with applications from hidden Markov models to quantum machine learningcited by 55
  3. 2016On the Latent Variable Interpretation in Sum-Product Networkscited by 40
  4. 2020Non-parametric Models for Non-negative Functionscited by 40
  5. 2021Tractable Regularization of Probabilistic Circuitscited by 39
  6. 2018Learning deep kernels for exponential family densitiescited by 36
  7. 2022Scaling Up Probabilistic Circuits by Latent Variable Distillationcited by 35
  8. 2016Parameterized Machine Learning for High-Energy Physicscited by 33
  9. 2021Tensor-Train Density Estimationcited by 30
  10. 2021Probabilistic Generating Circuitscited by 24
  11. 2021PSD Representations for Effective Probability Modelscited by 22
  12. 2020Probabilistic Circuits for Variational Inference in Discrete Graphical Modelscited by 20
  13. 2023How to Turn Your Knowledge Graph Embeddings into Generative Modelscited by 20
  14. 2016Visualizing and Understanding Sum-Product Networkscited by 18
  15. 2014On the Expressive Efficiency of Sum Product Networkscited by 16
  16. 2022Tractable Uncertainty for Structure Learningcited by 15
  17. 2023Squared Neural Families: A New Class of Tractable Density Modelscited by 12
  18. 2015Communication Complexity (for Algorithm Designers)cited by 5
  19. 2014Learning Negative Mixture Models by Tensor Decompositionscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 10; 2025: 17; 2026: 11.

39 papers.

Sort:

By area:

  1. 2024Scaling Tractable Probabilistic Circuits: A Systems Perspectivecited by 23
  2. 2024What is the Relationship between Tensor Factorizations and Circuits (and How Can We Exploit it)?cited by 22
  3. 2024Sum of Squares Circuitscited by 21
  4. 2024On the Relationship Between Monotone and Squared Probabilistic Circuitscited by 14
  5. 20253D Student Splatting and Scoopingcited by 14
  6. 2024Building Expressive and Tractable Probabilistic Generative Models: A Reviewcited by 12
  7. 2024EigenVI: score-based variational inference with orthogonal function expansionscited by 11
  8. 2025A Probabilistic Neuro-symbolic Layer for Algebraic Constraint Satisfactioncited by 11
  9. 2024Scaling Continuous Latent Variable Models as Probabilistic Integral Circuitscited by 9
  10. 2025Scaling Probabilistic Circuits via Monarch Matricescited by 9
  11. 2025Fast and Expressive Multi-Byte Prediction with Probabilistic Circuitscited by 9
  12. 2024Restructuring Tractable Probabilistic Circuitscited by 7
27 more
  1. 2025How to Square Tensor Networks and Circuits Without Squaring Themcited by 5
  2. 2025An extrapolated and provably convergent algorithm for nonlinear matrix decomposition with the ReLU functioncited by 4
  3. 2025Tractable Representation Learning with Probabilistic Circuitscited by 4
  4. 2025Tractable Sharpness-Aware Learning of Probabilistic Circuitscited by 3
  5. 2025PriorGuide: Test-Time Prior Adaptation for Simulation-Based Inferencecited by 3
  6. 2025Alternating Direction Method of Multipliers for Nonlinear Matrix Decompositionscited by 3
  7. 2026An Embarrassingly Simple Way to Optimize Orthogonal Matrices at Scalecited by 3
  8. 2023TERM Model: Tensor Ring Mixture Model for Density Estimationcited by 2
  9. 2024GraphSPNs: Sum-Product Networks Benefit From Canonical Orderingscited by 2
  10. 2024On Faster Marginalization with Squared Circuits via Orthonormalizationcited by 2
  11. 2025The Limits of Tractable Marginalizationcited by 2
  12. 2026Hellinger Multimodal Variational Autoencoderscited by 2
  13. 2026To Neuro-Symbolic Classification and Beyond by Compiling Description Logic Ontologies to Probabilistic Circuitscited by 2
  14. 2025Probabilistic Graph Circuits: Deep Generative Models for Tractable Probabilistic Inference over Graphscited by 1
  15. 2025Squared families: Searching beyond regular probability modelscited by 1
  16. 2025A Quantum Information Theoretic Approach to Tractable Probabilistic Modelscited by 1
  17. 2025Parsimonious Gaussian mixture models with piecewise-constant eigenvalue profilescited by 1
  18. 2026The Theory and Practice of MAP Inference over Non-Convex Constraintscited by 1
  19. 2026Geometry-Aware Probabilistic Circuits via Voronoi Tessellationscited by 1
  20. 2026A Sobering Look at Tabular Data Generation via Probabilistic Circuitscited by 1
  21. 2026ExplainFuzz: Explainable and Constraint-Conditioned Test Generation with Probabilistic Circuitscited by 1
  22. 2026How to Approximate Inference with Subtractive Mixture Modelscited by 1
  23. 2026Even More Guarantees for Variational Inference in the Presence of Symmetriescited by 1
  24. 2025Scalable Expectation Estimation with Subtractive Mixture Modelscited by 0
  25. 2025Learning Tractable Distributions Of Language Model Continuationscited by 0
  26. 2026Learning Markov Processes as Sum-of-Square Forms for Analytical Belief Propagationcited by 0
  27. 2026On the Complexity of Low-Rank Matrix Signing and Entrywise Power Matrix Factorizationcited by 0