# Tensor-Train Density Estimation

arXiv: https://arxiv.org/abs/2108.00089 · HTML: https://mlsanity.com/paper/2108.00089/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 29 references that are in the map.
- First posted on arXiv 30 Jul 2021.

## Builds on (most cited first, 15 of 15)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Pixel Recurrent Neural Networks](https://mlsanity.com/paper/1601.06759/) (2016)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models](https://mlsanity.com/paper/1810.01367/) (2018)
- [Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search](https://mlsanity.com/paper/2005.11129/) (2020)
- [On the Expressive Power of Deep Learning: A Tensor Analysis](https://mlsanity.com/paper/1509.05009/) (2015)
- [Block Neural Autoregressive Flow](https://mlsanity.com/paper/1904.04676/) (2019)
- [Expressive power of recurrent neural networks](https://mlsanity.com/paper/1711.00811/) (2017)
- [Nonparametric Density Estimation for High-Dimensional Data - Algorithms and Applications](https://mlsanity.com/paper/1904.00176/) (2019)
- [Tensors, Learning, and 'Kolmogorov Extension' for Finite-alphabet Random Vectors](https://mlsanity.com/paper/1712.00205/) (2017)
- [Low-rank Characteristic Tensor Density Estimation Part I: Foundations](https://mlsanity.com/paper/2008.12315/) (2020)
- [Learning Mixtures of Smooth Product Distributions: Identifiability and Algorithm](https://mlsanity.com/paper/1904.01156/) (2019)
- [Black-Box Autoregressive Density Estimation for State-Space Models](https://mlsanity.com/paper/1811.08337/) (2018)

## Built on by (most cited first, 30 of 30)

- [Subtractive Mixture Models via Squaring: Representation and Learning](https://mlsanity.com/paper/2310.00724/) (2023)
- [What is the Relationship between Tensor Factorizations and Circuits (and How Can We Exploit it)?](https://mlsanity.com/paper/2409.07953/) (2024)
- [How to Turn Your Knowledge Graph Embeddings into Generative Models](https://mlsanity.com/paper/2305.15944/) (2023)
- [On the Relationship Between Monotone and Squared Probabilistic Circuits](https://mlsanity.com/paper/2408.00876/) (2024)
- [Generative modeling via tensor train sketching](https://mlsanity.com/paper/2202.11788/) (2022)
- [Permutation Search of Tensor Network Structures via Local Sampling](https://mlsanity.com/paper/2206.06597/) (2022)
- [EigenVI: score-based variational inference with orthogonal function expansions](https://mlsanity.com/paper/2410.24054/) (2024)
- [Generative Learning of Continuous Data by Tensor Networks](https://mlsanity.com/paper/2310.20498/) (2023)
- [Generative Modeling via Hierarchical Tensor Sketching](https://mlsanity.com/paper/2304.05305/) (2023)
- [Guaranteed Nonconvex Factorization Approach for Tensor Train Recovery](https://mlsanity.com/paper/2401.02592/) (2024)
- [SHAP Meets Tensor Networks: Provably Tractable Explanations with Parallelism](https://mlsanity.com/paper/2510.21599/) (2025)
- [Language Modeling Using Tensor Trains](https://mlsanity.com/paper/2405.04590/) (2024)
- [How to Square Tensor Networks and Circuits Without Squaring Them](https://mlsanity.com/paper/2512.17090/) (2025)
- [High-dimensional density estimation with tensorizing flow](https://mlsanity.com/paper/2212.00759/) (2022)
- [Coarse-To-Fine Tensor Trains for Compact Visual Representations](https://mlsanity.com/paper/2406.04332/) (2024)
- [Sample Complexity Using Infinite Multiview Models](https://mlsanity.com/paper/2302.04292/) (2023)
- [Efficient Nonparametric Tensor Decomposition for Binary and Count Data](https://mlsanity.com/paper/2401.07711/) (2024)
- [Tensor Train for Global Optimization Problems in Robotics](https://mlsanity.com/paper/2206.05077/) (2022)
- [TERM Model: Tensor Ring Mixture Model for Density Estimation](https://mlsanity.com/paper/2312.08075/) (2023)
- [On Faster Marginalization with Squared Circuits via Orthonormalization](https://mlsanity.com/paper/2412.07883/) (2024)
- [Error Analysis of Tensor-Train Cross Approximation](https://mlsanity.com/paper/2207.04327/) (2022)
- [Deep importance sampling using tensor trains with application to a priori and a posteriori rare event estimation](https://mlsanity.com/paper/2209.01941/) (2022)
- [Tensorizing flows: a tool for variational inference](https://mlsanity.com/paper/2305.02460/) (2023)
- [Tensor Network Estimation of Distribution Algorithms](https://mlsanity.com/paper/2412.19780/) (2024)
- [Counting Phases and Faces Using Bayesian Thermodynamic Integration](https://mlsanity.com/paper/2206.07494/) (2022)
- [Tensor-Train Point Cloud Compression and Efficient Approximate Nearest-Neighbor Search](https://mlsanity.com/paper/2410.04462/) (2024)
- [Joint Bayesian Parameter and Model Order Estimation for Low-Rank Probability Mass Tensors](https://mlsanity.com/paper/2410.06329/) (2024)
- [Explaining Anomalies with Tensor Networks](https://mlsanity.com/paper/2505.03911/) (2025)
- [On the Approximation Complexity of Matrix Product Operator (MPO) Born Machines](https://mlsanity.com/paper/2605.11471/) (2026)
- [QuanVI: Score-based Variational Inference via Quantum Maximally Mixed States](https://mlsanity.com/paper/2609.39164/) (2026)

## Abstract (arXiv)

Estimation of probability density function from samples is one of the central problems in statistics and machine learning. Modern neural network-based models can learn high dimensional distributions but have problems with hyperparameter selection and are often prone to instabilities during training and inference. We propose a new efficient tensor train-based model for density estimation (TTDE). Such density parametrization allows exact sampling, calculation of cumulative and marginal density functions, and partition function. It also has very intuitive hyperparameters. We develop an efficient non-adversarial training procedure for TTDE based on the Riemannian optimization. Experimental results demonstrate the competitive performance of the proposed method in density estimation and sampling tasks, while TTDE significantly outperforms competitors in training speed.