# Learning with Subset Stacking

arXiv: https://arxiv.org/abs/2112.06251 · HTML: https://mlsanity.com/paper/2112.06251/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 45 references that are in the map.
- First posted on arXiv 12 Dec 2021.

## Builds on (most cited first, 5 of 5)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Keystroke Dynamics as Part of Lifelogging](https://mlsanity.com/paper/2010.10144/) (2020)
- [Generalized Robust Bayesian Committee Machine for Large-scale Gaussian Process Regression](https://mlsanity.com/paper/1806.00720/) (2018)
- [Magging: maximin aggregation for inhomogeneous large-scale data](https://mlsanity.com/paper/1409.2638/) (2014)

## Built on by (most cited first, 1 of 1)

- [Optimization with Dynamic Constraint Learning (DCL)](https://mlsanity.com/paper/2607.25719/) (2026)

## Abstract (arXiv)

We propose a new regression algorithm that learns from a set of input-output pairs. Our algorithm is designed for populations where the relation between the input variables and the output variable exhibits a heterogeneous behavior across the predictor space. The algorithm starts with generating subsets that are concentrated around random points in the input space. This is followed by training a local predictor for each subset. Those predictors are then combined in a novel way to yield an overall predictor. We call this algorithm "LEarning with Subset Stacking" or LESS, due to its resemblance to the method of stacking regressors. We offer bagging and boosting variants of LESS and test against the state-of-the-art methods on several datasets. Our comparison shows that LESS is highly competitive.