# Multi-Study Boosting: Theoretical Considerations for Merging vs. Ensembling

arXiv: https://arxiv.org/abs/2207.04588 · HTML: https://mlsanity.com/paper/2207.04588/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 39 references that are in the map.
- First posted on arXiv 11 Jul 2022.

## Builds on (most cited first, 4 of 4)

- [Exact post-selection inference, with application to the lasso](https://mlsanity.com/paper/1311.6238/) (2013)
- [Exact Post-Selection Inference for Sequential Regression Procedures](https://mlsanity.com/paper/1401.3889/) (2014)
- [A New Perspective on Boosting in Linear Regression via Subgradient Optimization and Relatives](https://mlsanity.com/paper/1505.04243/) (2015)
- [Cross-validation Approaches for Multi-study Predictions](https://mlsanity.com/paper/2007.12807/) (2020)

## Built on by (most cited first, 1 of 1)

- [Multi-Study R-Learner for Estimating Heterogeneous Treatment Effects Across Studies Using Statistical Machine Learning](https://mlsanity.com/paper/2306.01086/) (2023)

## Abstract (arXiv)

Cross-study replicability is a powerful model evaluation criterion that emphasizes generalizability of predictions. When training cross-study replicable prediction models, it is critical to decide between merging and treating the studies separately. We study boosting algorithms in the presence of potential heterogeneity in predictor-outcome relationships across studies and compare two multi-study learning strategies: 1) merging all the studies and training a single model, and 2) multi-study ensembling, which involves training a separate model on each study and ensembling the resulting predictions. In the regression setting, we provide theoretical guidelines based on an analytical transition point to determine whether it is more beneficial to merge or to ensemble for boosting with linear learners. In addition, we characterize a bias-variance decomposition of estimation error for boosting with component-wise linear learners. We verify the theoretical transition point result in simulation and illustrate how it can guide the decision on merging vs. ensembling in an application to breast cancer gene expression data.