mlsanityData to 7 Oct 2026

Probabilistic Integral Circuits

arXiv 2310.16986 ↗First posted 25 Oct 2023cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Continuous latent variables (LVs) are a key ingredient of many generative models, as they allow modelling expressive mixtures with an uncountable number of components. In contrast, probabilistic circuits (PCs) are hierarchical discrete mixtures represented as computational graphs composed of input, sum and product units. Unlike continuous LV models, PCs provide tractable inference but are limited to discrete LVs with categorical (i.e. unordered) states. We bridge these model classes by introducing probabilistic integral circuits (PICs), a new language of computational graphs that extends PCs with integral units representing continuous LVs. In the first place, PICs are symbolic computational graphs and are fully tractable in simple cases where analytical integration is possible. In practice, we parameterise PICs with light-weight neural nets delivering an intractable hierarchical continuous mixture that can be approximated arbitrarily well with large PCs using numerical quadrature. On several distribution estimation benchmarks, we show that such PIC-approximating PCs systematically outperform PCs commonly learned via expectation-maximization or SGD.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 28 of its 65 references that are in the map. First posted on arXiv 25 Oct 2023.

Builds on

28 of 65 references.

References · 28

  1. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  2. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  4. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  5. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  6. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 251
  7. 2019Practical Lossless Compression with Latent Variables using Bits Back Codingcited by 85
  8. 2019Integer Discrete Flows and Lossless Compressioncited by 82
  9. 2019Discrete Flows: Invertible Generative Models of Discrete Datacited by 68
  10. 2019Latent Normalizing Flows for Discrete Sequencescited by 66
  11. 2019Bit-Swap: Recursive Bits-Back Coding for Lossless Compression with Hierarchical Latent Variablescited by 51
  12. 2016On the Latent Variable Interpretation in Sum-Product Networkscited by 40
16 more
  1. 2021Tractable Regularization of Probabilistic Circuitscited by 39
  2. 2022Scaling Up Probabilistic Circuits by Latent Variable Distillationcited by 35
  3. 2016A Unified Approach for Learning the Parameters of Sum-Product Networkscited by 24
  4. 2019Conditional Sum-Product Networks: Imposing Structure on Deep Probabilistic Architecturescited by 24
  5. 2019Bayesian Learning of Sum-Product Networkscited by 24
  6. 2019Autoregressive Energy Machinescited by 23
  7. 2021Lossless Compression with Probabilistic Circuitscited by 20
  8. 2021Improving Lossless Compression Rates via Monte Carlo Bits-Back Codingcited by 19
  9. 2022Continuous Mixtures of Tractable Probabilistic Modelscited by 19
  10. 2023Understanding the Distillation Process from Deep Generative Models to Tractable Probabilistic Circuitscited by 19
  11. 2022Sparse Probabilistic Circuits via Pruning and Growingcited by 17
  12. 2021HyperSPNs: Compact and Expressive Probabilistic Circuitscited by 14
  13. 2023Squared Neural Families: A New Class of Tractable Density Modelscited by 12
  14. 2021A Compositional Atlas of Tractable Circuit Operations: From Simple Transformations to Complex Information-Theoretic Queriescited by 7
  15. 2023Bayesian Structure Scores for Probabilistic Circuitscited by 7
  16. 2019Sum-Product Network Decompilationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 3.

8 papers.

  1. 2024Scaling Tractable Probabilistic Circuits: A Systems Perspectivecited by 23
  2. 2024What is the Relationship between Tensor Factorizations and Circuits (and How Can We Exploit it)?cited by 22
  3. 2024Scaling Continuous Latent Variable Models as Probabilistic Integral Circuitscited by 9
  4. 2025Fast and Expressive Multi-Byte Prediction with Probabilistic Circuitscited by 9
  5. 2025Tractable Representation Learning with Probabilistic Circuitscited by 4
  6. 2026To Neuro-Symbolic Classification and Beyond by Compiling Description Logic Ontologies to Probabilistic Circuitscited by 2
  7. 2026Geometry-Aware Probabilistic Circuits via Voronoi Tessellationscited by 1
  8. 2026A Sobering Look at Tabular Data Generation via Probabilistic Circuitscited by 1