# Adaptive transfer learning

arXiv: https://arxiv.org/abs/2106.04455 · HTML: https://mlsanity.com/paper/2106.04455/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 70 references that are in the map.
- First posted on arXiv 8 Jun 2021.

## Builds on (most cited first, 20 of 20)

- [Detecting and Correcting for Label Shift with Black Box Predictors](https://mlsanity.com/paper/1802.03916/) (2018)
- [Certifying Some Distributional Robustness with Principled Adversarial Training](https://mlsanity.com/paper/1710.10571/) (2017)
- [The Benefit of Multitask Representation Learning](https://mlsanity.com/paper/1505.06279/) (2015)
- [Domain Generalization by Marginal Transfer Learning](https://mlsanity.com/paper/1711.07910/) (2017)
- [Classification with Asymmetric Label Noise: Consistency and Maximal Denoising](https://mlsanity.com/paper/1303.1208/) (2013)
- [Marginal Singularity, and the Benefits of Labels in Covariate-Shift](https://mlsanity.com/paper/1803.01833/) (2018)
- [Transfer Learning for Nonparametric Classification: Minimax Rate and Adaptive Classifier](https://mlsanity.com/paper/1906.02903/) (2019)
- [On the Value of Target Data in Transfer Learning](https://mlsanity.com/paper/2002.04747/) (2020)
- [Risk Bounds for the Majority Vote: From a PAC-Bayesian Analysis to a Learning Algorithm](https://mlsanity.com/paper/1503.08329/) (2015)
- [Rates of Convergence for Nearest Neighbor Classification](https://mlsanity.com/paper/1407.0067/) (2014)
- [A Generalized Neyman-Pearson Criterion for Optimal Domain Adaptation](https://mlsanity.com/paper/1810.01545/) (2018)
- [Minimax optimal approaches to the label shift problem in non-parametric settings](https://mlsanity.com/paper/2003.10443/) (2020)
- [Local nearest neighbour classification with applications to semi-supervised learning](https://mlsanity.com/paper/1704.00642/) (2017)
- [Recent progress in log-concave density estimation](https://mlsanity.com/paper/1709.03154/) (2017)
- [Fast Rates for a kNN Classifier Robust to Unknown Asymmetric Label Noise](https://mlsanity.com/paper/1906.04542/) (2019)
- [Classification with unknown class-conditional label noise on non-compact feature spaces](https://mlsanity.com/paper/1902.05627/) (2019)
- [Learning from Multiple Corrupted Sources, with Application to Learning from Label Proportions](https://mlsanity.com/paper/1910.04665/) (2019)
- [Minimax rates for cost-sensitive learning on manifolds with approximate nearest neighbours](https://mlsanity.com/paper/1803.00310/) (2018)
- [Cellular Tree Classifiers](https://mlsanity.com/paper/1301.4679/) (2013)
- [Causality in cognitive neuroscience: concepts, challenges, and distributional robustness](https://mlsanity.com/paper/2002.06060/) (2020)

## Built on by (most cited first, 4 of 4)

- [Model-free Test Time Adaptation for Out-Of-Distribution Detection](https://mlsanity.com/paper/2311.16420/) (2023)
- [Transfer Learning with Uncertainty Quantification: Random Effect Calibration of Source to Target (RECaST)](https://mlsanity.com/paper/2211.16557/) (2022)
- [Minimax And Adaptive Transfer Learning for Nonparametric Classification under Distributed Differential Privacy Constraints](https://mlsanity.com/paper/2406.20088/) (2024)
- [Personalizing black-box models for nonparametric regression with minimax optimality](https://mlsanity.com/paper/2601.01432/) (2026)

## Abstract (arXiv)

In transfer learning, we wish to make inference about a target population when we have access to data both from the distribution itself, and from a different but related source distribution. We introduce a flexible framework for transfer learning in the context of binary classification, allowing for covariate-dependent relationships between the source and target distributions that are not required to preserve the Bayes decision boundary. Our main contributions are to derive the minimax optimal rates of convergence (up to poly-logarithmic factors) in this problem, and show that the optimal rate can be achieved by an algorithm that adapts to key aspects of the unknown transfer relationship, as well as the smoothness and tail parameters of our distributional classes. This optimal rate turns out to have several regimes, depending on the interplay between the relative sample sizes and the strength of the transfer relationship, and our algorithm achieves optimality by careful, decision tree-based calibration of local nearest-neighbour procedures.