# Leveraging Probabilistic Circuits for Nonparametric Multi-Output Regression

arXiv: https://arxiv.org/abs/2106.08687 · HTML: https://mlsanity.com/paper/2106.08687/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 45 references that are in the map.
- First posted on arXiv 16 Jun 2021.

## Builds on (most cited first, 13 of 13)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [When Gaussian Process Meets Big Data: A Review of Scalable GPs](https://mlsanity.com/paper/1807.01065/) (2018)
- [Einsum Networks: Fast and Scalable Learning of Tractable Probabilistic Circuits](https://mlsanity.com/paper/2004.06231/) (2020)
- [A Differential Approach to Inference in Bayesian Networks](https://mlsanity.com/paper/1301.3847/) (2013)
- [Distributed Gaussian Processes](https://mlsanity.com/paper/1502.02843/) (2015)
- [Approximate Kernel-based Conditional Independence Tests for Fast Non-Parametric Causal Discovery](https://mlsanity.com/paper/1702.03877/) (2017)
- [Heterogeneous Multi-output Gaussian Process Prediction](https://mlsanity.com/paper/1805.07633/) (2018)
- [Scalable Exact Inference in Multi-Output Gaussian Processes](https://mlsanity.com/paper/1911.06287/) (2019)
- [Conditional Sum-Product Networks: Imposing Structure on Deep Probabilistic Architectures](https://mlsanity.com/paper/1905.08550/) (2019)
- [Bayesian Learning of Sum-Product Networks](https://mlsanity.com/paper/1905.10884/) (2019)
- [Deep Structured Mixtures of Gaussian Processes](https://mlsanity.com/paper/1910.04536/) (2019)
- [Embarrassingly Parallel Inference for Gaussian Processes](https://mlsanity.com/paper/1702.08420/) (2017)
- [Dynamic Sum Product Networks for Tractable Inference on Sequence Data (Extended Version)](https://mlsanity.com/paper/1511.04412/) (2015)

## Built on by (most cited first, 8 of 8)

- [Scalable Neural-Probabilistic Answer Set Programming](https://mlsanity.com/paper/2306.08397/) (2023)
- [Building Expressive and Tractable Probabilistic Generative Models: A Review](https://mlsanity.com/paper/2402.00759/) (2024)
- [Solving Marginal MAP Exactly by Probabilistic Circuit Transformations](https://mlsanity.com/paper/2111.04833/) (2021)
- [Characteristic Circuits](https://mlsanity.com/paper/2312.07790/) (2023)
- [ExplainerPFN: Towards tabular foundation models for model-free zero-shot feature importance estimations](https://mlsanity.com/paper/2601.23068/) (2026)
- [Regularized Multi-output Gaussian Convolution Process with Domain Adaptation](https://mlsanity.com/paper/2409.02778/) (2024)
- [Sum-Product-Attention Networks: Leveraging Self-Attention in Probabilistic Circuits](https://mlsanity.com/paper/2109.06587/) (2021)
- [TreeFlow: Going beyond Tree-based Gaussian Probabilistic Regression](https://mlsanity.com/paper/2206.04140/) (2022)

## Abstract (arXiv)

Inspired by recent advances in the field of expert-based approximations of Gaussian processes (GPs), we present an expert-based approach to large-scale multi-output regression using single-output GP experts. Employing a deeply structured mixture of single-output GPs encoded via a probabilistic circuit allows us to capture correlations between multiple output dimensions accurately. By recursively partitioning the covariate space and the output space, posterior inference in our model reduces to inference on single-output GP experts, which only need to be conditioned on a small subset of the observations. We show that inference can be performed exactly and efficiently in our model, that it can capture correlations between output dimensions and, hence, often outperforms approaches that do not incorporate inter-output correlations, as demonstrated on several data sets in terms of the negative log predictive density.