# Optimisation of Overparametrized Sum-Product Networks

arXiv: https://arxiv.org/abs/1905.08196 · HTML: https://mlsanity.com/paper/1905.08196/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 May 2019.

## Built on by (most cited first, 1 of 1)

- [Top-Down Bayesian Posterior Sampling for Sum-Product Networks](https://mlsanity.com/paper/2406.12353/) (2024)

## Abstract (arXiv)

It seems to be a pearl of conventional wisdom that parameter learning in deep sum-product networks is surprisingly fast compared to shallow mixture models. This paper examines the effects of overparameterization in sum-product networks on the speed of parameter optimisation. Using theoretical analysis and empirical experiments, we show that deep sum-product networks exhibit an implicit acceleration compared to their shallow counterpart. In fact, gradient-based optimisation in deep tree-structured sum-product networks is equal to gradient ascend with adaptive and time-varying learning rates and additional momentum terms.