mlsanityData to 7 Oct 2026

What is the Relationship between Tensor Factorizations and Circuits (and How Can We Exploit it)?

arXiv 2409.07953 ↗First posted 12 Sep 2024cs.LGcited by 22

Not on the map. Why?

Abstract (arXiv)

This paper establishes a rigorous connection between circuit representations and tensor factorizations, two seemingly distinct yet fundamentally related areas. By connecting these fields, we highlight a series of opportunities that can benefit both communities. Our work generalizes popular tensor factorizations within the circuit language, and unifies various circuit learning algorithms under a single, generalized hierarchical factorization framework. Specifically, we introduce a modular "Lego block" approach to build tensorized circuit architectures. This, in turn, allows us to systematically construct and explore various circuit and tensor factorization models while maintaining tractability. This connection not only clarifies similarities and differences in existing models, but also enables the development of a comprehensive pipeline for building and optimizing new circuit/tensor factorization architectures. We show the effectiveness of our framework through extensive empirical evaluations, and highlight new research opportunities for tensor factorizations in probabilistic modeling.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 56 of its 146 references that are in the map. First posted on arXiv 12 Sep 2024.

Builds on

56 of 146 references.

References · 56

  1. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  2. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  3. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
  4. 2015A Review of Relational Machine Learning for Knowledge Graphscited by 264
  5. 2020A Comprehensive Survey of Neural Architecture Search: Challenges and Solutionscited by 253
  6. 2016Tensor Decomposition for Signal Processing and Machine Learningcited by 223
  7. 2022Semantic Probabilistic Layers for Neuro-Symbolic Learningcited by 103
  8. 2020Einsum Networks: Fast and Scalable Learning of Tractable Probabilistic Circuitscited by 102
  9. 2015On the Expressive Power of Deep Learning: A Tensor Analysiscited by 97
  10. 2013A Differential Approach to Inference in Bayesian Networkscited by 93
  11. 2017Unsupervised Generative Modeling Using Matrix Product Statescited by 87
  12. 2019Practical Lossless Compression with Latent Variables using Bits Back Codingcited by 85
38 more
  1. 2019A Tensorized Transformer for Language Modelingcited by 85
  2. 2019Integer Discrete Flows and Lossless Compressioncited by 82
  3. 2023Tractable Control for Autoregressive Language Generationcited by 65
  4. 2019Tree Tensor Networks for Generative Modelingcited by 63
  5. 2023Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcutscited by 63
  6. 2019Expressive power of tensor-network factorizations for probabilistic modeling, with applications from hidden Markov models to quantum machine learningcited by 55
  7. 2021Tensor Methods in Computer Vision and Deep Learningcited by 52
  8. 2019Bit-Swap: Recursive Bits-Back Coding for Lossless Compression with Hierarchical Latent Variablescited by 51
  9. 2016On the Latent Variable Interpretation in Sum-Product Networkscited by 40
  10. 2021Tractable Regularization of Probabilistic Circuitscited by 39
  11. 2023Subtractive Mixture Models via Squaring: Representation and Learningcited by 39
  12. 2024BEARS Make Neuro-Symbolic Models Aware of their Reasoning Shortcutscited by 36
  13. 2022Scaling Up Probabilistic Circuits by Latent Variable Distillationcited by 35
  14. 2018From probabilistic graphical models to generalized tensor networks for supervised learningcited by 31
  15. 2021Tensor-Train Density Estimationcited by 30
  16. 2016A Unified Approach for Learning the Parameters of Sum-Product Networkscited by 24
  17. 2019Conditional Sum-Product Networks: Imposing Structure on Deep Probabilistic Architecturescited by 24
  18. 2024Sum of Squares Circuitscited by 21
  19. 2021Lossless Compression with Probabilistic Circuitscited by 20
  20. 2023How to Turn Your Knowledge Graph Embeddings into Generative Modelscited by 20
  21. 2021Improving Lossless Compression Rates via Monte Carlo Bits-Back Codingcited by 19
  22. 2022Continuous Mixtures of Tractable Probabilistic Modelscited by 19
  23. 2023Understanding the Distillation Process from Deep Generative Models to Tractable Probabilistic Circuitscited by 19
  24. 2016Visualizing and Understanding Sum-Product Networkscited by 18
  25. 2024On the Independence Assumption in Neurosymbolic Learningcited by 18
  26. 2022Sparse Probabilistic Circuits via Pruning and Growingcited by 17
  27. 2014On the Expressive Efficiency of Sum Product Networkscited by 16
  28. 2017Duality of Graphical Models and Tensor Networkscited by 15
  29. 2020Low-rank Characteristic Tensor Density Estimation Part I: Foundationscited by 15
  30. 2024Building Expressive and Tractable Probabilistic Generative Models: A Reviewcited by 12
  31. 2020Scaling up Hybrid Probabilistic Inference with Logical and Arithmetic Constraints via Message Passingcited by 11
  32. 2014Positive semidefinite rankcited by 10
  33. 2023Knowledge Graph Embeddings in the Biomedical Domain: Are They Useful? A Look at Link Prediction, Rule Learning, and Downstream Polypharmacy Taskscited by 9
  34. 2024Scaling Continuous Latent Variable Models as Probabilistic Integral Circuitscited by 9
  35. 2021The Tensor Brain: A Unified Theory of Perception, Memory and Semantic Decodingcited by 8
  36. 2023Probabilistic Integral Circuitscited by 8
  37. 2023Characteristic Circuitscited by 8
  38. 2014The square root rank of the correlation polytope is exponentialcited by 4

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 13; 2026: 8.

22 papers.

Sort:

  1. 2025A Probabilistic Neuro-symbolic Layer for Algebraic Constraint Satisfactioncited by 11
  2. 2025Scaling Probabilistic Circuits via Monarch Matricescited by 9
  3. 2025Fast and Expressive Multi-Byte Prediction with Probabilistic Circuitscited by 9
  4. 2026Mitigating Bias in Locally Constrained Decoding via Tractable Proposalscited by 8
  5. 2024Restructuring Tractable Probabilistic Circuitscited by 7
  6. 2025Compositionality Unlocks Deep Interpretable Modelscited by 6
  7. 2025How to Square Tensor Networks and Circuits Without Squaring Themcited by 5
  8. 2025Rethinking Probabilistic Circuit Parameter Learningcited by 4
  9. 2025Tractable Representation Learning with Probabilistic Circuitscited by 4
  10. 2025Deep Polynomial Chaos Expansioncited by 3
  11. 2025Tractable Sharpness-Aware Learning of Probabilistic Circuitscited by 3
  12. 2026To Neuro-Symbolic Classification and Beyond by Compiling Description Logic Ontologies to Probabilistic Circuitscited by 2
10 more
  1. 2026Near-Universal Multiplicative Updates for Nonnegative Einsum Factorizationcited by 2
  2. 2025Probabilistic Graph Circuits: Deep Generative Models for Tractable Probabilistic Inference over Graphscited by 1
  3. 2025A Quantum Information Theoretic Approach to Tractable Probabilistic Modelscited by 1
  4. 2026Geometry-Aware Probabilistic Circuits via Voronoi Tessellationscited by 1
  5. 2026A Sobering Look at Tabular Data Generation via Probabilistic Circuitscited by 1
  6. 2026How to Approximate Inference with Subtractive Mixture Modelscited by 1
  7. 2025Scalable Expectation Estimation with Subtractive Mixture Modelscited by 0
  8. 2025Efficient Probabilistic Tensor Networkscited by 0
  9. 2026On the Approximation Complexity of Matrix Product Operator Born Machinescited by 0
  10. 2026The Expressivity Boundary of Probabilistic Circuits: A Comparison with Large Language Modelscited by 0