mlsanityData to 7 Oct 2026

Einsum Networks: Fast and Scalable Learning of Tractable Probabilistic Circuits

arXiv 2004.06231 ↗First posted 13 Apr 2020cs.LG stat.MLcited by 102

Not on the map. Why?

Abstract (arXiv)

Probabilistic circuits (PCs) are a promising avenue for probabilistic modeling, as they permit a wide range of exact and efficient inference routines. Recent ``deep-learning-style'' implementations of PCs strive for a better scalability, but are still difficult to train on real-world data, due to their sparsely connected computational graphs. In this paper, we propose Einsum Networks (EiNets), a novel implementation design for PCs, improving prior art in several regards. At their core, EiNets combine a large number of arithmetic operations in a single monolithic einsum-operation, leading to speedups and memory savings of up to two orders of magnitude, in comparison to previous implementations. As an algorithmic contribution, we show that the implementation of Expectation-Maximization (EM) can be simplified for PCs, by leveraging automatic differentiation. Furthermore, we demonstrate that EiNets scale well to datasets which were previously out of reach, such as SVHN and CelebA, and that they can be used as faithful generative image models.

Key facts. Cited by 102 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 17; 2022: 17; 2023: 18; 2024: 18; 2025: 22; 2026: 10.

102 papers.

Sort:

By area:

  1. 2022Semantic Probabilistic Layers for Neuro-Symbolic Learningcited by 103
  2. 2022An Introduction to Neural Data Compressioncited by 78
  3. 2023Tractable Control for Autoregressive Language Generationcited by 65
  4. 2022Adversarial random forests for density estimation and generative modelingcited by 44
  5. 2021Tractable Regularization of Probabilistic Circuitscited by 39
  6. 2023Subtractive Mixture Models via Squaring: Representation and Learningcited by 39
  7. 2022Scaling Up Probabilistic Circuits by Latent Variable Distillationcited by 35
  8. 2021On the Complexity of SHAP-Score-Based Explanations: Tractability via Knowledge Compilation and Non-Approximability Resultscited by 32
  9. 2022Neuro-Symbolic Entropy Regularizationcited by 27
  10. 2021Probabilistic Generating Circuitscited by 24
  11. 2024Scaling Tractable Probabilistic Circuits: A Systems Perspectivecited by 23
  12. 2021Interventional Sum-Product Networks: Causal Inference with Tractable Probabilistic Modelscited by 22
38 more
  1. 2023Image Inpainting via Tractable Steering of Diffusion Modelscited by 22
  2. 2024What is the Relationship between Tensor Factorizations and Circuits (and How Can We Exploit it)?cited by 22
  3. 2023A Pseudo-Semantic Loss for Autoregressive Models with Logical Constraintscited by 21
  4. 2021Lossless Compression with Probabilistic Circuitscited by 20
  5. 2022Continuous Mixtures of Tractable Probabilistic Modelscited by 19
  6. 2023Understanding the Distillation Process from Deep Generative Models to Tractable Probabilistic Circuitscited by 19
  7. 2022Sparse Probabilistic Circuits via Pruning and Growingcited by 17
  8. 2024KLay: Accelerating Arithmetic Circuits for Neurosymbolic AIcited by 16
  9. 2021HyperSPNs: Compact and Expressive Probabilistic Circuitscited by 14
  10. 2023Scalable Neural-Probabilistic Answer Set Programmingcited by 14
  11. 2024On the Relationship Between Monotone and Squared Probabilistic Circuitscited by 14
  12. 2023Semantic Strengthening of Neuro-Symbolic Learningcited by 12
  13. 2024Building Expressive and Tractable Probabilistic Generative Models: A Reviewcited by 12
  14. 2024Polynomial Semantics of Tractable Probabilistic Circuitscited by 12
  15. 2021Solving Marginal MAP Exactly by Probabilistic Circuit Transformationscited by 11
  16. 2022Neural Graphical Modelscited by 11
  17. 2024Controllable Generation via Locally Constrained Resamplingcited by 11
  18. 2022Dynamic Programming in Rank Space: Scaling Structured Inference with Low-Rank HMMs and PCFGscited by 10
  19. 2023Compositional Probabilistic and Causal Inference using Tractable Circuit Modelscited by 10
  20. 2021Handling Epistemic and Aleatory Uncertainties in Probabilistic Circuitscited by 9
  21. 2022Tractable Boolean and Arithmetic Circuitscited by 9
  22. 2024Scaling Continuous Latent Variable Models as Probabilistic Integral Circuitscited by 9
  23. 2025Scaling Probabilistic Circuits via Monarch Matricescited by 9
  24. 2025Fast and Expressive Multi-Byte Prediction with Probabilistic Circuitscited by 9
  25. 2021Leveraging Probabilistic Circuits for Nonparametric Multi-Output Regressioncited by 8
  26. 2021SLASH: Embracing Probabilistic Circuits into Neural Answer Set Programmingcited by 8
  27. 2023End-to-End Supervised Multilabel Contrastive Learningcited by 8
  28. 2023Characteristic Circuitscited by 8
  29. 2025TRACE Back from the Future: A Probabilistic Reasoning Approach to Controllable Language Generationcited by 8
  30. 2021A Compositional Atlas of Tractable Circuit Operations: From Simple Transformations to Complex Information-Theoretic Queriescited by 7
  31. 2022Unsupervised Discontinuous Constituency Parsing with Mildly Context-Sensitive Grammarscited by 7
  32. 2023Probabilistic Circuits That Know What They Don't Knowcited by 7
  33. 2023Bayesian Structure Scores for Probabilistic Circuitscited by 7
  34. 2024Probabilistic Neural Circuitscited by 7
  35. 2024Restructuring Tractable Probabilistic Circuitscited by 7
  36. 2025Neural Probabilistic Circuits: Enabling Compositional and Interpretable Predictions through Logical Reasoningcited by 6
  37. 2024Credibility-Aware Multi-Modal Fusion Using Probabilistic Circuitscited by 5
  38. 2024A Unified Framework for Human-Allied Learning of Probabilistic Circuitscited by 5

50 most cited of 102 All 102 (JSON)