mlsanityData to 14 Oct 2026

How to Square Tensor Networks and Circuits Without Squaring Them

arXiv 2512.17090 ↗First posted 18 Dec 2025cs.LG cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Squared tensor networks (TNs) and their extension as computational graphs--squared circuits--have been used as expressive distribution estimators, yet supporting closed-form marginalization. However, the squaring operation introduces additional complexity when computing the partition function or marginalizing variables, which hinders their applicability in ML. To solve this issue, canonical forms of TNs are parameterized via unitary matrices to simplify the computation of marginals. However, these canonical forms do not apply to circuits, as they can represent factorizations that do not directly map to a known TN. Inspired by the ideas of orthogonality in canonical forms and determinism in circuits enabling tractable maximization, we show how to parameterize squared circuits to overcome their marginalization overhead. Our parameterizations unlock efficient marginalization even in factorizations different from TNs, but encoded as circuits, whose structure would otherwise make marginalization computationally hard. Finally, our experiments on distribution estimation show how our proposed conditions in squared circuits come with no expressiveness loss, while enabling more efficient learning.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 57 of its 129 references that are in the map. First posted on arXiv 18 Dec 2025.

Builds on

57 of 129 references.

References · 57

  1. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  2. 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 772
  3. 2015Unitary Evolution Recurrent Neural Networkscited by 294
  4. 2019Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Groupcited by 134
  5. 2016Tensor Ring Decompositioncited by 127
  6. 2020Geoopt: Riemannian Optimization in PyTorchcited by 114
  7. 2020Einsum Networks: Fast and Scalable Learning of Tractable Probabilistic Circuitscited by 103
  8. 2022Semantic Probabilistic Layers for Neuro-Symbolic Learningcited by 103
  9. 2013A Differential Approach to Inference in Bayesian Networkscited by 93
  10. 2017Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networkscited by 92
  11. 2017Unsupervised Generative Modeling Using Matrix Product Statescited by 88
  12. 2019Trivializations for Gradient-Based Optimization on Manifoldscited by 79
38 more
  1. 2022An Introduction to Neural Data Compressioncited by 79
  2. 2019Tree Tensor Networks for Generative Modelingcited by 64
  3. 2023Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcutscited by 63
  4. 2019Expressive power of tensor-network factorizations for probabilistic modeling, with applications from hidden Markov models to quantum machine learningcited by 55
  5. 2020Non-parametric Models for Non-negative Functionscited by 41
  6. 2021Tractable Regularization of Probabilistic Circuitscited by 39
  7. 2023Subtractive Mixture Models via Squaring: Representation and Learningcited by 39
  8. 2024BEARS Make Neuro-Symbolic Models Aware of their Reasoning Shortcutscited by 37
  9. 2021Fast and accurate optimization on the orthogonal manifold without retractioncited by 35
  10. 2022Scaling Up Probabilistic Circuits by Latent Variable Distillationcited by 35
  11. 2018From probabilistic graphical models to generalized tensor networks for supervised learningcited by 32
  12. 2024CompactifAI: Extreme Compression of Large Language Models using Quantum-Inspired Tensor Networkscited by 32
  13. 2021Tensor-Train Density Estimationcited by 30
  14. 2021Probabilistic Generating Circuitscited by 25
  15. 2021PSD Representations for Effective Probability Modelscited by 23
  16. 2020Group Fairness by Probabilistic Modeling with Latent Fair Decisionscited by 22
  17. 2024What is the Relationship between Tensor Factorizations and Circuits (and How Can We Exploit it)?cited by 22
  18. 2024Sum of Squares Circuitscited by 21
  19. 2020Probabilistic Circuits for Variational Inference in Discrete Graphical Modelscited by 20
  20. 2021Lossless Compression with Probabilistic Circuitscited by 20
  21. 2023How to Turn Your Knowledge Graph Embeddings into Generative Modelscited by 20
  22. 2019On Tractable Computation of Expected Predictionscited by 19
  23. 2016Visualizing and Understanding Sum-Product Networkscited by 18
  24. 2022VectorAdam for Rotation Equivariant Geometry Optimizationcited by 18
  25. 2023Infeasible Deterministic, Stochastic, and Variance-Reduction Algorithms for Optimization under Orthogonality Constraintscited by 18
  26. 2014On the Expressive Efficiency of Sum Product Networkscited by 16
  27. 2024On the Relationship Between Monotone and Squared Probabilistic Circuitscited by 14
  28. 2023Squared Neural Families: A New Class of Tractable Density Modelscited by 12
  29. 2024Polynomial Semantics of Tractable Probabilistic Circuitscited by 12
  30. 2024EigenVI: score-based variational inference with orthogonal function expansionscited by 11
  31. 2025A Probabilistic Neuro-symbolic Layer for Algebraic Constraint Satisfactioncited by 11
  32. 2024A Compositional Atlas for Algebraic Circuitscited by 10
  33. 2023Generative Learning of Continuous Data by Tensor Networkscited by 9
  34. 2024Scaling Continuous Latent Variable Models as Probabilistic Integral Circuitscited by 9
  35. 2025Scaling Probabilistic Circuits via Monarch Matricescited by 9
  36. 2020On the Relationship Between Probabilistic Circuits and Determinantal Point Processescited by 8
  37. 2023Characteristic Circuitscited by 8
  38. 2020Tangent-Space Gradient Optimization of Tensor Network for Machine Learningcited by 7

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Hellinger Multimodal Variational Autoencoderscited by 3
  2. 2026An Embarrassingly Simple Way to Optimize Orthogonal Matrices at Scalecited by 3
  3. 2026The Theory and Practice of MAP Inference over Non-Convex Constraintscited by 1
  4. 2026A Sobering Look at Tabular Data Generation via Probabilistic Circuitscited by 1
  5. 2026How to Approximate Inference with Subtractive Mixture Modelscited by 1