# Proxy-informed Bayesian transfer learning with unknown sources

arXiv: https://arxiv.org/abs/2411.03263 · HTML: https://mlsanity.com/paper/2411.03263/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 40 references that are in the map.
- First posted on arXiv 5 Nov 2024.

## Builds on (most cited first, 16 of 16)

- [Characterizing and Avoiding Negative Transfer](https://mlsanity.com/paper/1811.09751/) (2018)
- [A Survey of Human-in-the-loop for Machine Learning](https://mlsanity.com/paper/2108.00941/) (2021)
- [Bayesian Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.03836/) (2018)
- [Recasting Gradient-Based Meta-Learning as Hierarchical Bayes](https://mlsanity.com/paper/1801.08930/) (2018)
- [Modern Bayesian Experimental Design](https://mlsanity.com/paper/2302.14545/) (2023)
- [Meta-Learning Probabilistic Inference For Prediction](https://mlsanity.com/paper/1805.09921/) (2018)
- [A survey on domain adaptation theory: learning bounds and theoretical guarantees](https://mlsanity.com/paper/2004.11829/) (2020)
- [An Introduction to Proximal Causal Learning](https://mlsanity.com/paper/2009.10982/) (2020)
- [Learning to select data for transfer learning with Bayesian Optimization](https://mlsanity.com/paper/1707.05246/) (2017)
- [Robust Bayesian inference via coarsening](https://mlsanity.com/paper/1506.06101/) (2015)
- [Adapting to Latent Subgroup Shifts via Concepts and Proxies](https://mlsanity.com/paper/2212.11254/) (2022)
- [Improving drug sensitivity predictions in precision medicine through active expert knowledge elicitation](https://mlsanity.com/paper/1705.03290/) (2017)
- [Markov Chain Importance Sampling -- a highly efficient estimator for MCMC](https://mlsanity.com/paper/1805.07179/) (2018)
- [Robustifying likelihoods by optimistically re-weighting data](https://mlsanity.com/paper/2303.10525/) (2023)
- [Bayesian Active Learning in the Presence of Nuisance Parameters](https://mlsanity.com/paper/2310.14968/) (2023)
- [Proxy Methods for Domain Adaptation](https://mlsanity.com/paper/2403.07442/) (2024)

## Built on by (most cited first, 2 of 2)

- [Epistemic Errors of Imperfect Multitask Learners When Distributions Shift](https://mlsanity.com/paper/2505.23496/) (2025)
- [Point-Identification of a Robust Predictor Under Latent Shift with Imperfect Proxies](https://mlsanity.com/paper/2603.15158/) (2026)

## Abstract (arXiv)

Generalization outside the scope of one's training data requires leveraging prior knowledge about the effects that transfer, and the effects that don't, between different data sources. Transfer learning is a framework for specifying and refining this knowledge about sets of source (training) and target (prediction) data. A challenging open problem is addressing the empirical phenomenon of negative transfer, whereby the transfer learner performs worse on the target data after taking the source data into account than before. We first introduce a Bayesian perspective on negative transfer, and then a method to address it. The key insight from our formulation is that negative transfer can stem from misspecified prior information about non-transferable causes of the source data. Our proposed method, proxy-informed robust method for probabilistic transfer learning (PROMPT), does not require prior knowledge of the source data (the data sources may be "unknown"). PROMPT is thus applicable when differences between tasks are unobserved, such as in the presence of latent confounders. Moreover, the learner need not have access to observations in the target task (may not have the ability to "fine-tune"), and instead makes use of proxy (indirect) information. Our theoretical results show that the threat of negative transfer does not depend on the informativeness of the proxy information, highlighting the usefulness of PROMPT in cases where only noisy indirect information, such as human feedback, is available.