# Learning Shared Kernel Models: the Shared Kernel EM algorithm

arXiv: https://arxiv.org/abs/2205.09041 · HTML: https://mlsanity.com/paper/2205.09041/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 44 references that are in the map.
- First posted on arXiv 15 May 2022.

## Builds on (most cited first, 1 of 1)

- [From the Expectation Maximisation Algorithm to Autoencoded Variational Bayes](https://mlsanity.com/paper/2010.13551/) (2020)

## Built on by (most cited first, 1 of 1)

- [Improvements to Supervised EM Learning of Shared Kernel Models by Feature Space Partitioning](https://mlsanity.com/paper/2205.15304/) (2022)

## Abstract (arXiv)

Expectation maximisation (EM) is an unsupervised learning method for estimating the parameters of a finite mixture distribution. It works by introducing "hidden" or "latent" variables via Baum's auxiliary function Q that allow the joint data likelihood to be expressed as a product of simple factors. The relevance of EM has increased since the introduction of the variational lower bound (VLB): the VLB differs from Baum's auxiliary function only by the entropy of the PDF of the latent variables Z. We first present a rederivation of the standard EM algorithm using data association ideas from the field of multiple target tracking, using K-valued scalar data association hypotheses rather than the usual binary indicator vectors. The same method is then applied to a little known but much more general type of supervised EM algorithm for shared kernel models, related to probabilistic radial basis function networks. We address a number of shortcomings in the derivations that have been published previously in this area. In particular, we give theoretically rigorous derivations of (i) the complete data likelihood; (ii) Baum's auxiliary function (the E-step) and (iii) the maximisation (M-step) in the case of Gaussian shared kernel models. The subsequent algorithm, called shared kernel EM (SKEM), is then applied to a digit recognition problem using a novel 7-segment digit representation. Variants of the algorithm that use different numbers of features and different EM algorithm dimensions are compared in terms of mean accuracy and mean IoU. A simplified classifier is proposed that decomposes the joint data PDF as a product of lower order PDFs over non-overlapping subsets of variables. The effect of different numbers of assumed mixture components K is also investigated. High-level source code for the data generation and SKEM algorithm is provided.