# Synthetic Model Combination: An Instance-wise Approach to Unsupervised Ensemble Learning

arXiv: https://arxiv.org/abs/2210.05320 · HTML: https://mlsanity.com/paper/2210.05320/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 62 references that are in the map.
- First posted on arXiv 11 Oct 2022.

## Builds on (most cited first, 16 of 16)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Likelihood Ratios for Out-of-Distribution Detection](https://mlsanity.com/paper/1906.02845/) (2019)
- [How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models](https://mlsanity.com/paper/2102.08921/) (2021)
- [A review of domain adaptation without target labels](https://mlsanity.com/paper/1901.05335/) (2019)
- [Uncertainty Quantification and Deep Ensembles](https://mlsanity.com/paper/2007.08792/) (2020)
- [Scalable Bayesian Inverse Reinforcement Learning](https://mlsanity.com/paper/2102.06483/) (2021)
- [Dangers of Bayesian Model Averaging under Covariate Shift](https://mlsanity.com/paper/2106.11905/) (2021)
- [Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift](https://mlsanity.com/paper/2006.14988/) (2020)
- [Unsupervised Ensemble Learning with Dependent Classifiers](https://mlsanity.com/paper/1510.05830/) (2015)
- [Optimal Bayesian Transfer Learning](https://mlsanity.com/paper/1801.00857/) (2018)
- [POETREE: Interpretable Policy Learning with Adaptive Decision Trees](https://mlsanity.com/paper/2203.08057/) (2022)
- [A Deep Learning Approach to Unsupervised Ensemble Learning](https://mlsanity.com/paper/1602.02285/) (2016)
- [The Medkit-Learn(ing) Environment: Medical Decision Modelling through Simulation](https://mlsanity.com/paper/2106.04240/) (2021)
- [Inverse Online Learning: Understanding Non-Stationary and Reactionary Policies](https://mlsanity.com/paper/2203.07338/) (2022)
- [Unsupervised Ensemble Regression](https://mlsanity.com/paper/1703.02965/) (2017)

## Built on by (most cited first, 2 of 2)

- [Input Adaptive Bayesian Model Averaging](https://mlsanity.com/paper/2510.22054/) (2025)
- [Unsupervised Ensemble Learning Through Deep Energy-based Models](https://mlsanity.com/paper/2601.20556/) (2026)

## Abstract (arXiv)

Consider making a prediction over new test data without any opportunity to learn from a training set of labelled data - instead given access to a set of expert models and their predictions alongside some limited information about the dataset used to train them. In scenarios from finance to the medical sciences, and even consumer practice, stakeholders have developed models on private data they either cannot, or do not want to, share. Given the value and legislation surrounding personal information, it is not surprising that only the models, and not the data, will be released - the pertinent question becoming: how best to use these models? Previous work has focused on global model selection or ensembling, with the result of a single final model across the feature space. Machine learning models perform notoriously poorly on data outside their training domain however, and so we argue that when ensembling models the weightings for individual instances must reflect their respective domains - in other words models that are more likely to have seen information on that instance should have more attention paid to them. We introduce a method for such an instance-wise ensembling of models, including a novel representation learning step for handling sparse high-dimensional domains. Finally, we demonstrate the need and generalisability of our method on classical machine learning tasks as well as highlighting a real world use case in the pharmacological setting of vancomycin precision dosing.