# Inference at the data's edge: Gaussian processes for modeling and inference under model-dependency, poor overlap, and extrapolation

arXiv: https://arxiv.org/abs/2407.10442 · HTML: https://mlsanity.com/paper/2407.10442/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 30 references that are in the map.
- First posted on arXiv 15 Jul 2024.

## Builds on (most cited first, 7 of 7)

- [Gaussian Process Kernels for Pattern Discovery and Extrapolation](https://mlsanity.com/paper/1302.4245/) (2013)
- [Regression Discontinuity Designs Using Covariates](https://mlsanity.com/paper/1809.03904/) (2018)
- [A Nonparametric Bayesian Methodology for Regression Discontinuity Designs](https://mlsanity.com/paper/1704.04858/) (2017)
- [Estimating the effects of a California gun control program with Multitask Gaussian Processes](https://mlsanity.com/paper/2110.07006/) (2021)
- [On the reliability of published findings using the regression discontinuity design in political science](https://mlsanity.com/paper/2109.14526/) (2021)
- [Kpop: A kernel balancing approach for reducing specification assumptions in survey weighting](https://mlsanity.com/paper/2107.08075/) (2021)
- [Generalized Kernel Regularized Least Squares](https://mlsanity.com/paper/2209.14355/) (2022)

## Abstract (arXiv)

Many inferential tasks involve fitting models to observed data and predicting outcomes at new covariate values, requiring interpolation or extrapolation. Conventional methods select a single best-fitting model, discarding fits that were similarly plausible in-sample but would yield sharply different predictions out-of-sample. Gaussian Processes (GPs) offer a principled alternative. Rather than committing to one conditional expectation function, GPs deliver a posterior distribution over outcomes at any covariate value. This posterior effectively retains the range of models consistent with the data, widening uncertainty intervals where extrapolation magnifies divergence. In this way, the GP's uncertainty estimates reflect the implications of extrapolation on our predictions, helping to tame the "dangers of extreme counterfactuals" (King & Zeng, 2006). The approach requires (i) specifying a covariance function linking outcome similarity to covariate similarity, and (ii) assuming Gaussian noise around the conditional expectation. We provide an accessible introduction to GPs with emphasis on this property, along with a simple, automated procedure for hyperparameter selection implemented in the R package gpss. We illustrate the value of GPs for capturing counterfactual uncertainty in three settings: (i) treatment effect estimation with poor overlap, (ii) interrupted time series requiring extrapolation beyond pre-intervention data, and (iii) regression discontinuity designs where estimates hinge on boundary behavior.