# ISTRBoost: Importance Sampling Transfer Regression using Boosting

arXiv: https://arxiv.org/abs/2204.12044 · HTML: https://mlsanity.com/paper/2204.12044/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 73 references that are in the map.
- First posted on arXiv 26 Apr 2022.

## Builds on (most cited first, 20 of 20)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [A Survey on Deep Transfer Learning](https://mlsanity.com/paper/1808.01974/) (2018)
- [Not All Samples Are Created Equal: Deep Learning with Importance Sampling](https://mlsanity.com/paper/1803.00942/) (2018)
- [Domain Generalization by Marginal Transfer Learning](https://mlsanity.com/paper/1711.07910/) (2017)
- [Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis](https://mlsanity.com/paper/1804.06353/) (2018)
- [Stochastic Optimization with Importance Sampling](https://mlsanity.com/paper/1401.2753/) (2014)
- [Importance Weighted Adversarial Nets for Partial Domain Adaptation](https://mlsanity.com/paper/1803.09210/) (2018)
- [Learning from Multiple Cities: A Meta-Learning Approach for Spatial-Temporal Prediction](https://mlsanity.com/paper/1901.08518/) (2019)
- [Machine Learning for the Geosciences: Challenges and Opportunities](https://mlsanity.com/paper/1711.04708/) (2017)
- [Domain Adaptive Transfer Learning with Specialist Models](https://mlsanity.com/paper/1811.07056/) (2018)
- [Hypothesis Transfer Learning via Transformation Functions](https://mlsanity.com/paper/1612.01020/) (2016)
- [Task Transfer by Preference-Based Cost Learning](https://mlsanity.com/paper/1805.04686/) (2018)
- [Learning an Invariant Hilbert Space for Domain Adaptation](https://mlsanity.com/paper/1611.08350/) (2016)
- [Deep Air Learning: Interpolation, Prediction, and Feature Analysis of Fine-grained Air Quality](https://mlsanity.com/paper/1711.00939/) (2017)
- [Spectral Metric for Dataset Complexity Assessment](https://mlsanity.com/paper/1905.07299/) (2019)
- [Instance-based Deep Transfer Learning](https://mlsanity.com/paper/1809.02776/) (2018)
- [Transfer Learning for Linear Regression: a Statistical Test of Gain](https://mlsanity.com/paper/2102.09504/) (2021)
- [Gradient Importance Sampling](https://mlsanity.com/paper/1507.05781/) (2015)

## Abstract (arXiv)

Current Instance Transfer Learning (ITL) methodologies use domain adaptation and sub-space transformation to achieve successful transfer learning. However, these methodologies, in their processes, sometimes overfit on the target dataset or suffer from negative transfer if the test dataset has a high variance. Boosting methodologies have been shown to reduce the risk of overfitting by iteratively re-weighing instances with high-residual. However, this balance is usually achieved with parameter optimization, as well as reducing the skewness in weights produced due to the size of the source dataset. While the former can be achieved, the latter is more challenging and can lead to negative transfer. We introduce a simpler and more robust fix to this problem by building upon the popular boosting ITL regression methodology, two-stage TrAdaBoost.R2. Our methodology,~us, is a boosting and random-forest based ensemble methodology that utilizes importance sampling to reduce the skewness due to the source dataset. We show that~us~performs better than competitive transfer learning methodologies 63% of the time. It also displays consistency in its performance over diverse datasets with varying complexities, as opposed to the sporadic results observed for other transfer learning methodologies.