# Ensemble Prediction via Covariate-dependent Stacking

arXiv: https://arxiv.org/abs/2408.09755 · HTML: https://mlsanity.com/paper/2408.09755/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 93 references that are in the map.
- First posted on arXiv 19 Aug 2024.

## Builds on (most cited first, 6 of 6)

- [Dynamic Bayesian Predictive Synthesis in Time Series Forecasting](https://mlsanity.com/paper/1601.07463/) (2016)
- [Model Averaging and its Use in Economics](https://mlsanity.com/paper/1709.08221/) (2017)
- [Additive stacking for disaggregate electricity demand forecasting](https://mlsanity.com/paper/2005.10092/) (2020)
- [Bayesian Causal Synthesis for Meta-Inference on Heterogeneous Treatment Effects](https://mlsanity.com/paper/2304.07726/) (2023)
- [Process-based Inference for Spatial Energetics Using Bayesian Predictive Stacking](https://mlsanity.com/paper/2405.09906/) (2024)
- [Bayesian Spatial Predictive Synthesis](https://mlsanity.com/paper/2203.05197/) (2022)

## Built on by (most cited first, 1 of 1)

- [When Does Dynamic Ensembling Pay Off? Diagnosing Regionwise Gains in Regression under Distribution Shift](https://mlsanity.com/paper/2608.18330/) (2026)

## Abstract (arXiv)

This study proposes a novel approach to ensemble prediction, called "covariate-dependent stacking" (CDST). Unlike traditional stacking and model averaging methods, CDST allows model weights to vary flexibly as a function of covariates, thereby enhancing predictive performance in complex scenarios. We formulate the covariate-dependent weights through combinations of basis functions and estimate them via cross-validation optimization. To analyze the theoretical properties, we establish an oracle inequality regarding the expected loss to be minimized for estimating model weights. Through comprehensive simulation studies and an application to large-scale land price prediction, we demonstrate that the CDST consistently outperforms conventional model averaging methods, particularly on datasets where base models fail to capture the underlying complexity. Our findings suggest that the CDST is especially valuable for, but not limited to, spatio-temporal prediction problems, offering a powerful tool for researchers and practitioners across a wide spectrum of data analysis fields.