# Joint Bayesian Parameter and Model Order Estimation for Low-Rank Probability Mass Tensors

arXiv: https://arxiv.org/abs/2410.06329 · HTML: https://mlsanity.com/paper/2410.06329/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 53 references that are in the map.
- First posted on arXiv 8 Oct 2024.

## Builds on (most cited first, 14 of 14)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Tensor Decomposition for Signal Processing and Machine Learning](https://mlsanity.com/paper/1607.01668/) (2016)
- [Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination](https://mlsanity.com/paper/1401.6497/) (2014)
- [Expressive power of tensor-network factorizations for probabilistic modeling, with applications from hidden Markov models to quantum machine learning](https://mlsanity.com/paper/1907.03741/) (2019)
- [From probabilistic graphical models to generalized tensor networks for supervised learning](https://mlsanity.com/paper/1806.05964/) (2018)
- [Tensor-Train Density Estimation](https://mlsanity.com/paper/2108.00089/) (2021)
- [Tensor networks for unsupervised machine learning](https://mlsanity.com/paper/2106.12974/) (2021)
- [Tensors, Learning, and 'Kolmogorov Extension' for Finite-alphabet Random Vectors](https://mlsanity.com/paper/1712.00205/) (2017)
- [Learning Mixtures of Smooth Product Distributions: Identifiability and Algorithm](https://mlsanity.com/paper/1904.01156/) (2019)
- [Rethinking Bayesian Learning for Data Analysis: The Art of Prior and Inference in Sparsity-Aware Modeling](https://mlsanity.com/paper/2205.14283/) (2022)
- [Recovering Joint Probability of Discrete Random Variables from Pairwise Marginals](https://mlsanity.com/paper/2006.16912/) (2020)
- [Towards Flexible Sparsity-Aware Modeling: Automatic Tensor Rank Learning Using The Generalized Hyperbolic Prior](https://mlsanity.com/paper/2009.02472/) (2020)
- [Bayesian Conditional Tensor Factorizations for High-Dimensional Classification](https://mlsanity.com/paper/1301.4950/) (2013)
- [Bayesian factorizations of big sparse tensors](https://mlsanity.com/paper/1306.1598/) (2013)

## Abstract (arXiv)

Obtaining a reliable estimate of the joint probability mass function (PMF) of a set of random variables from observed data is a significant objective in statistical signal processing and machine learning. Modelling the joint PMF as a tensor that admits a low-rank canonical polyadic decomposition (CPD) has enabled the development of efficient PMF estimation algorithms. However, these algorithms require the rank (model order) of the tensor to be specified beforehand. In real-world applications, the true rank is unknown. Therefore, an appropriate rank is usually selected from a candidate set either by observing validation errors or by computing various likelihood-based information criteria, a procedure that could be costly in terms of computational time or hardware resources, or could result in mismatched models which affect the model accuracy. This paper presents a novel Bayesian framework for estimating the low-rank components of a joint PMF tensor and simultaneously inferring its rank from the observed data. We specify a Bayesian PMF estimation model and employ appropriate prior distributions for the model parameters, allowing the rank to be inferred without cross-validation.We then derive a deterministic solution based on variational inference (VI) to approximate the posterior distributions of various model parameters. Numerical experiments involving both synthetic data and real classification and item recommendation data illustrate the advantages of our VI-based method in terms of estimation accuracy, automatic rank detection, and computational efficiency.