# Sequential Monte Carlo Learning for Time Series Structure Discovery

arXiv: https://arxiv.org/abs/2307.09607 · HTML: https://mlsanity.com/paper/2307.09607/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 64 references that are in the map.
- First posted on arXiv 13 Jul 2023.

## Builds on (most cited first, 20 of 20)

- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [Structure Discovery in Nonparametric Regression through Compositional Kernel Search](https://mlsanity.com/paper/1302.4922/) (2013)
- [Bayesian Symbolic Regression](https://mlsanity.com/paper/1910.08892/) (2019)
- [Automatic Construction and Natural-Language Description of Nonparametric Regression Models](https://mlsanity.com/paper/1402.4304/) (2014)
- [Spatio-Temporal Variational Gaussian Processes](https://mlsanity.com/paper/2111.01732/) (2021)
- [Scaling up the Automatic Statistician: Scalable Structure Discovery using Gaussian Processes](https://mlsanity.com/paper/1706.02524/) (2017)
- [Bayesian Synthesis of Probabilistic Programs for Automatic Data Modeling](https://mlsanity.com/paper/1907.06249/) (2019)
- [Involutive MCMC: a Unifying Framework](https://mlsanity.com/paper/2006.16653/) (2020)
- [Time series forecasting with Gaussian Processes needs priors](https://mlsanity.com/paper/2009.08102/) (2020)
- [Variational Combinatorial Sequential Monte Carlo Methods for Bayesian Phylogenetic Inference](https://mlsanity.com/paper/2106.00075/) (2021)
- [Recursive Monte Carlo and Variational Inference with Auxiliary Variables](https://mlsanity.com/paper/2203.02836/) (2022)
- [Combining Pseudo-Point and State Space Approximations for Sum-Separable Gaussian Processes](https://mlsanity.com/paper/2106.10210/) (2021)
- [CrossCat: A Fully Bayesian Nonparametric Method for Analyzing Heterogeneous, High Dimensional Data](https://mlsanity.com/paper/1512.01272/) (2015)
- [Sequential Monte Carlo for Graphical Models](https://mlsanity.com/paper/1402.0330/) (2014)
- [Time Series Structure Discovery via Probabilistic Program Synthesis](https://mlsanity.com/paper/1611.07051/) (2016)
- [Estimators of Entropy and Information via Inference in Probabilistic Models](https://mlsanity.com/paper/2202.12363/) (2022)
- [Hierarchical Infinite Relational Model](https://mlsanity.com/paper/2108.07208/) (2021)
- [Scaling Nonparametric Bayesian Inference via Subsample-Annealing](https://mlsanity.com/paper/1402.5473/) (2014)
- [Gaussian Process Kernels for Popular State-Space Time Series Models](https://mlsanity.com/paper/1610.08074/) (2016)
- [Probabilistic structure discovery in time series data](https://mlsanity.com/paper/1611.06863/) (2016)

## Built on by (most cited first, 5 of 5)

- [Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems](https://mlsanity.com/paper/2405.06624/) (2024)
- [Automated Statistical Model Discovery with Language Models](https://mlsanity.com/paper/2402.17879/) (2024)
- [Mamba time series forecasting with uncertainty quantification](https://mlsanity.com/paper/2503.10873/) (2025)
- [Worst-Case Analysis is Maximum-A-Posteriori Estimation](https://mlsanity.com/paper/2310.09774/) (2023)
- [Incremental Structure Discovery of Classification via Sequential Monte Carlo](https://mlsanity.com/paper/2408.07875/) (2024)

## Abstract (arXiv)

This paper presents a new approach to automatically discovering accurate models of complex time series data. Working within a Bayesian nonparametric prior over a symbolic space of Gaussian process time series models, we present a novel structure learning algorithm that integrates sequential Monte Carlo (SMC) and involutive MCMC for highly effective posterior inference. Our method can be used both in "online" settings, where new data is incorporated sequentially in time, and in "offline" settings, by using nested subsets of historical data to anneal the posterior. Empirical measurements on real-world time series show that our method can deliver 10x--100x runtime speedups over previous MCMC and greedy-search structure learning algorithms targeting the same model family. We use our method to perform the first large-scale evaluation of Gaussian process time series structure learning on a prominent benchmark of 1,428 econometric datasets. The results show that our method discovers sensible models that deliver more accurate point forecasts and interval forecasts over multiple horizons as compared to widely used statistical and neural baselines that struggle on this challenging data.