# Incremental Structure Discovery of Classification via Sequential Monte Carlo

arXiv: https://arxiv.org/abs/2408.07875 · HTML: https://mlsanity.com/paper/2408.07875/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 31 references that are in the map.
- First posted on arXiv 15 Aug 2024.

## Builds on (most cited first, 10 of 10)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Structure Discovery in Nonparametric Regression through Compositional Kernel Search](https://mlsanity.com/paper/1302.4922/) (2013)
- [Streaming Sparse Gaussian Process Approximations](https://mlsanity.com/paper/1705.07131/) (2017)
- [An invitation to sequential Monte Carlo samplers](https://mlsanity.com/paper/2007.11936/) (2020)
- [Random Feature-based Online Multi-kernel Learning in Environments with Unknown Dynamics](https://mlsanity.com/paper/1712.09983/) (2017)
- [Inference Networks for Sequential Monte Carlo in Graphical Models](https://mlsanity.com/paper/1602.06701/) (2016)
- [Bayesian Synthesis of Probabilistic Programs for Automatic Data Modeling](https://mlsanity.com/paper/1907.06249/) (2019)
- [Involutive MCMC: a Unifying Framework](https://mlsanity.com/paper/2006.16653/) (2020)
- [Incremental Ensemble Gaussian Processes](https://mlsanity.com/paper/2110.06777/) (2021)
- [Sequential Monte Carlo Learning for Time Series Structure Discovery](https://mlsanity.com/paper/2307.09607/) (2023)

## Abstract (arXiv)

Gaussian Processes (GPs) provide a powerful framework for making predictions and understanding uncertainty for classification with kernels and Bayesian non-parametric learning. Building such models typically requires strong prior knowledge to define preselect kernels, which could be ineffective for online applications of classification that sequentially process data because features of data may shift during the process. To alleviate the requirement of prior knowledge used in GPs and learn new features from data that arrive successively, this paper presents a novel method to automatically discover models of classification on complex data with little prior knowledge. Our method adapts a recently proposed technique for GP-based time-series structure discovery, which integrates GPs and Sequential Monte Carlo (SMC). We extend the technique to handle extra latent variables in GP classification, such that our method can effectively and adaptively learn a-priori unknown structures of classification from continuous input. In addition, our method adapts new batch of data with updated structures of models. Our experiments show that our method is able to automatically incorporate various features of kernels on synthesized data and real-world data for classification. In the experiments of real-world data, our method outperforms various classification methods on both online and offline setting achieving a 10% accuracy improvement on one benchmark.