# Automatic Construction and Natural-Language Description of Nonparametric Regression Models

arXiv: https://arxiv.org/abs/1402.4304 · HTML: https://mlsanity.com/paper/1402.4304/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Feb 2014.

## Built on by (most cited first, 30 of 40)

- [Priors in Bayesian Deep Learning: A Review](https://mlsanity.com/paper/2105.06868/) (2021)
- [The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learning](https://mlsanity.com/paper/2304.05366/) (2023)
- [Bayesian Model Selection, the Marginal Likelihood, and Generalization](https://mlsanity.com/paper/2202.11678/) (2022)
- [Automated Statistical Model Discovery with Language Models](https://mlsanity.com/paper/2402.17879/) (2024)
- [A Primer on Bayesian Neural Networks: Review and Debates](https://mlsanity.com/paper/2309.16314/) (2023)
- [Automating Data Science: Prospects and Challenges](https://mlsanity.com/paper/2105.05699/) (2021)
- [Adaptive Kernel Design for Bayesian Optimization Is a Piece of CAKE with LLMs](https://mlsanity.com/paper/2509.17998/) (2025)
- [Generating Computational Cognitive Models using Large Language Models](https://mlsanity.com/paper/2502.00879/) (2025)
- [Slow Momentum with Fast Reversion: A Trading Strategy Using Deep Learning and Changepoint Detection](https://mlsanity.com/paper/2105.13727/) (2021)
- [Amortized Inference for Gaussian Process Hyperparameters of Structured Kernels](https://mlsanity.com/paper/2306.09819/) (2023)
- [Structural Kernel Search via Bayesian Optimization and Symbolical Optimal Transport](https://mlsanity.com/paper/2210.11836/) (2022)
- [Tisane: Authoring Statistical Models via Formal Reasoning from Conceptual and Data Relationships](https://mlsanity.com/paper/2201.02705/) (2022)
- [Online Time Series Anomaly Detection with State Space Gaussian Processes](https://mlsanity.com/paper/2201.06763/) (2022)
- [Sequential Monte Carlo Learning for Time Series Structure Discovery](https://mlsanity.com/paper/2307.09607/) (2023)
- [Hierarchical-Hyperplane Kernels for Actively Learning Gaussian Process Models of Nonstationary Systems](https://mlsanity.com/paper/2303.10022/) (2023)
- [The Minecraft Kernel: Modelling correlated Gaussian Processes in the Fourier domain](https://mlsanity.com/paper/2103.06950/) (2021)
- [Gaussian Process Latent Variable Modeling for Few-shot Time Series Forecasting](https://mlsanity.com/paper/2212.10306/) (2022)
- [Automated Model Discovery via Multi-modal & Multi-step Pipeline](https://mlsanity.com/paper/2509.25946/) (2025)
- [Model Selection for Production System via Automated Online Experiments](https://mlsanity.com/paper/2105.13420/) (2021)
- [On the Identifiability and Interpretability of Gaussian Process Models](https://mlsanity.com/paper/2310.17023/) (2023)
- [Just One Layer Norm Guarantees Stable Extrapolation](https://mlsanity.com/paper/2505.14512/) (2025)
- [Gaussian Process Latent Class Choice Models](https://mlsanity.com/paper/2101.12252/) (2021)
- [Tractable Computation of Expected Kernels](https://mlsanity.com/paper/2102.10562/) (2021)
- [Volatility Based Kernels and Moving Average Means for Accurate Forecasting with Gaussian Processes](https://mlsanity.com/paper/2207.06544/) (2022)
- [Trust in Language Grounding: a new AI challenge for human-robot teams](https://mlsanity.com/paper/2209.02066/) (2022)
- [Beyond Intuition, a Framework for Applying GPs to Real-World Data](https://mlsanity.com/paper/2307.03093/) (2023)
- [Learning material synthesis-process-structure-property relationship by data fusion: Bayesian Coregionalization N-Dimensional Piecewise Function Learning](https://mlsanity.com/paper/2311.06228/) (2023)
- [Exploratory Visual Analysis for Increasing Data Readiness in Artificial Intelligence Projects](https://mlsanity.com/paper/2409.03805/) (2024)
- [VESTA: Visual Exploration with Statistical Tool Agents](https://mlsanity.com/paper/2606.00384/) (2026)
- [Online structural kernel selection for mobile health](https://mlsanity.com/paper/2107.09949/) (2021)

## Abstract (arXiv)

This paper presents the beginnings of an automatic statistician, focusing on regression problems. Our system explores an open-ended space of statistical models to discover a good explanation of a data set, and then produces a detailed report with figures and natural-language text. Our approach treats unknown regression functions nonparametrically using Gaussian processes, which has two important consequences. First, Gaussian processes can model functions in terms of high-level properties (e.g. smoothness, trends, periodicity, changepoints). Taken together with the compositional structure of our language of models this allows us to automatically describe functions in simple terms. Second, the use of flexible nonparametric models and a rich language for composing them in an open-ended manner also results in state-of-the-art extrapolation performance evaluated over 13 real time series data sets from various domains.