# Scaling up the Automatic Statistician: Scalable Structure Discovery using Gaussian Processes

arXiv: https://arxiv.org/abs/1706.02524 · HTML: https://mlsanity.com/paper/1706.02524/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jun 2017.

## Built on by (most cited first, 19 of 19)

- [Priors in Bayesian Deep Learning: A Review](https://mlsanity.com/paper/2105.06868/) (2021)
- [Surrogate modeling for Bayesian optimization beyond a single Gaussian process](https://mlsanity.com/paper/2205.14090/) (2022)
- [Tighter Bounds on the Log Marginal Likelihood of Gaussian Process Regression Using Conjugate Gradients](https://mlsanity.com/paper/2102.08314/) (2021)
- [Incremental Ensemble Gaussian Processes](https://mlsanity.com/paper/2110.06777/) (2021)
- [Constraining Gaussian Processes to Systems of Linear Ordinary Differential Equations](https://mlsanity.com/paper/2208.12515/) (2022)
- [Kernel Identification Through Transformers](https://mlsanity.com/paper/2106.08185/) (2021)
- [Rethinking Bayesian Learning for Data Analysis: The Art of Prior and Inference in Sparsity-Aware Modeling](https://mlsanity.com/paper/2205.14283/) (2022)
- [Trace Encoding in Process Mining: a survey and benchmarking](https://mlsanity.com/paper/2301.02167/) (2023)
- [Learning Energy Conserving Dynamics Efficiently with Hamiltonian Gaussian Processes](https://mlsanity.com/paper/2303.01925/) (2023)
- [Sequential Monte Carlo Learning for Time Series Structure Discovery](https://mlsanity.com/paper/2307.09607/) (2023)
- [Gaussian Process Latent Variable Modeling for Few-shot Time Series Forecasting](https://mlsanity.com/paper/2212.10306/) (2022)
- [Automated Model Discovery via Multi-modal & Multi-step Pipeline](https://mlsanity.com/paper/2509.25946/) (2025)
- [Model Selection for Production System via Automated Online Experiments](https://mlsanity.com/paper/2105.13420/) (2021)
- [Adaptive Cholesky Gaussian Processes](https://mlsanity.com/paper/2202.10769/) (2022)
- [Weighted Ensembles for Active Learning with Adaptivity](https://mlsanity.com/paper/2206.05009/) (2022)
- [Reconstructing Kernel-based Machine Learning Force Fields with Super-linear Convergence](https://mlsanity.com/paper/2212.12737/) (2022)
- [Recommendations for Baselines and Benchmarking Approximate Gaussian Processes](https://mlsanity.com/paper/2402.09849/) (2024)
- [Online structural kernel selection for mobile health](https://mlsanity.com/paper/2107.09949/) (2021)
- [On the Laplace Approximation as Model Selection Criterion for Gaussian Processes](https://mlsanity.com/paper/2403.09215/) (2024)

## Abstract (arXiv)

Automating statistical modelling is a challenging problem in artificial intelligence. The Automatic Statistician takes a first step in this direction, by employing a kernel search algorithm with Gaussian Processes (GP) to provide interpretable statistical models for regression problems. However this does not scale due to its O(N3) running time for the model selection. We propose Scalable Kernel Composition (SKC), a scalable kernel search algorithm that extends the Automatic Statistician to bigger data sets. In doing so, we derive a cheap upper bound on the GP marginal likelihood that sandwiches the marginal likelihood with the variational lower bound . We show that the upper bound is significantly tighter than the lower bound and thus useful for model selection.