# Scalable Random Feature Latent Variable Models

arXiv: https://arxiv.org/abs/2410.17700 · HTML: https://mlsanity.com/paper/2410.17700/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 61 references that are in the map.
- First posted on arXiv 23 Oct 2024.

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [Bayesian Deep Learning and a Probabilistic Perspective of Generalization](https://mlsanity.com/paper/2002.08791/) (2020)
- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [Advances in Variational Inference](https://mlsanity.com/paper/1711.05597/) (2017)
- [Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)](https://mlsanity.com/paper/1503.01057/) (2015)
- [Implicit Reparameterization Gradients](https://mlsanity.com/paper/1805.08498/) (2018)
- [Online Clustered Codebook](https://mlsanity.com/paper/2307.15139/) (2023)
- [Distributed Variational Representation Learning](https://mlsanity.com/paper/1807.04193/) (2018)
- [Block stochastic gradient iteration for convex and nonconvex optimization](https://mlsanity.com/paper/1408.2597/) (2014)
- [Rethinking Bayesian Learning for Data Analysis: The Art of Prior and Inference in Sparsity-Aware Modeling](https://mlsanity.com/paper/2205.14283/) (2022)
- [Bayesian Nonparametric Kernel-Learning](https://mlsanity.com/paper/1506.08776/) (2015)
- [Latent variable modeling with random features](https://mlsanity.com/paper/2006.11145/) (2020)
- [Representation Learning by Rotating Your Faces](https://mlsanity.com/paper/1705.11136/) (2017)
- [Harmonized Multimodal Learning with Gaussian Process Latent Variable Models](https://mlsanity.com/paper/1908.04979/) (2019)
- [ControlVAE: Tuning, Analytical Properties, and Performance Analysis](https://mlsanity.com/paper/2011.01754/) (2020)
- [Preventing Model Collapse in Gaussian Process Latent Variable Models](https://mlsanity.com/paper/2404.01697/) (2024)

## Abstract (arXiv)

Random feature latent variable models (RFLVMs) represent the state-of-the-art in latent variable models, capable of handling non-Gaussian likelihoods and effectively uncovering patterns in high-dimensional data. However, their heavy reliance on Monte Carlo sampling results in scalability issues which makes it difficult to use these models for datasets with a massive number of observations. To scale up RFLVMs, we turn to the optimization-based variational Bayesian inference (VBI) algorithm which is known for its scalability compared to sampling-based methods. However, implementing VBI for RFLVMs poses challenges, such as the lack of explicit probability distribution functions (PDFs) for the Dirichlet process (DP) in the kernel learning component, and the incompatibility of existing VBI algorithms with RFLVMs. To address these issues, we introduce a stick-breaking construction for DP to obtain an explicit PDF and a novel VBI algorithm called ``block coordinate descent variational inference" (BCD-VI). This enables the development of a scalable version of RFLVMs, or in short, SRFLVM. Our proposed method shows scalability, computational efficiency, superior performance in generating informative latent representations and the ability of imputing missing data across various real-world datasets, outperforming state-of-the-art competitors.