# Predicting with Proxies: Transfer Learning in High Dimension

arXiv: https://arxiv.org/abs/1812.11097 · HTML: https://mlsanity.com/paper/1812.11097/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Dec 2018.

## Built on by (most cited first, 24 of 24)

- [Transfer Learning under High-dimensional Generalized Linear Models](https://mlsanity.com/paper/2105.14328/) (2021)
- [Adaptive and Robust Multi-Task Learning](https://mlsanity.com/paper/2202.05250/) (2022)
- [Combining Observational and Randomized Data for Estimating Heterogeneous Treatment Effects](https://mlsanity.com/paper/2202.12891/) (2022)
- [Policy Learning with Adaptively Collected Data](https://mlsanity.com/paper/2105.02344/) (2021)
- [Targeting Underrepresented Populations in Precision Medicine: A Federated Transfer Learning Approach](https://mlsanity.com/paper/2108.12112/) (2021)
- [Algorithmic Fairness in Business Analytics: Directions for Research and Practice](https://mlsanity.com/paper/2207.10991/) (2022)
- [Identifying the Dynamics of a System by Leveraging Data from Similar Systems](https://mlsanity.com/paper/2204.05446/) (2022)
- [Multitask Learning and Bandits via Robust Statistics](https://mlsanity.com/paper/2112.14233/) (2021)
- [A Class of Geometric Structures in Transfer Learning: Minimax Bounds and Optimality](https://mlsanity.com/paper/2202.11685/) (2022)
- [Optimal Multitask Linear Regression and Contextual Bandits under Sparse Heterogeneity](https://mlsanity.com/paper/2306.06291/) (2023)
- [Group-Sparse Matrix Factorization for Transfer Learning of Word Embeddings](https://mlsanity.com/paper/2104.08928/) (2021)
- [Data-pooling Reinforcement Learning for Personalized Healthcare Intervention](https://mlsanity.com/paper/2211.08998/) (2022)
- [Robust Generalization of Quadratic Neural Networks via Function Identification](https://mlsanity.com/paper/2109.10935/) (2021)
- [Efficient and Multiply Robust Risk Estimation under General Forms of Dataset Shift](https://mlsanity.com/paper/2306.16406/) (2023)
- [Guided Adversarial Robust Transfer Learning with Source Mixing](https://mlsanity.com/paper/2309.06534/) (2023)
- [Transfer learning for tensor Gaussian graphical models](https://mlsanity.com/paper/2211.09391/) (2022)
- [Addressing distributional shifts in operations management: The case of order fulfillment in customized production](https://mlsanity.com/paper/2304.11910/) (2023)
- [The ART of Transfer Learning: An Adaptive and Robust Pipeline](https://mlsanity.com/paper/2305.00520/) (2023)
- [Assessing Electricity Service Unfairness with Transfer Counterfactual Learning](https://mlsanity.com/paper/2310.03258/) (2023)
- [Transfer Learning for Spatial Autoregressive Models with Application to U.S. Presidential Election Prediction](https://mlsanity.com/paper/2405.15600/) (2024)
- [Adaptive Lasso, Transfer Lasso, and Beyond: An Asymptotic Perspective](https://mlsanity.com/paper/2308.15838/) (2023)
- [Sparse Optimization for Transfer Learning: A L0-Regularized Framework for Multi-Source Domain Adaptation](https://mlsanity.com/paper/2504.04812/) (2025)
- [GeoERM: Geometry-Aware Multi-Task Representation Learning on Riemannian Manifolds](https://mlsanity.com/paper/2505.02972/) (2025)
- [Transfer Learning for Meta-analysis Under Covariate Shift](https://mlsanity.com/paper/2604.02656/) (2026)

## Abstract (arXiv)

Predictive analytics is increasingly used to guide decision-making in many applications. However, in practice, we often have limited data on the true predictive task of interest, and must instead rely on more abundant data on a closely-related proxy predictive task. For example, e-commerce platforms use abundant customer click data (proxy) to make product recommendations rather than the relatively sparse customer purchase data (true outcome of interest); alternatively, hospitals often rely on medical risk scores trained on a different patient population (proxy) rather than their own patient population (true cohort of interest) to assign interventions. Yet, not accounting for the bias in the proxy can lead to sub-optimal decisions. Using real datasets, we find that this bias can often be captured by a sparse function of the features. Thus, we propose a novel two-step estimator that uses techniques from high-dimensional statistics to efficiently combine a large amount of proxy data and a small amount of true data. We prove upper bounds on the error of our proposed estimator and lower bounds on several heuristics used by data scientists; in particular, our proposed estimator can achieve the same accuracy with exponentially less true data (in the number of features). Our proof relies on a new LASSO tail inequality for approximately sparse vectors. Finally, we demonstrate the effectiveness of our approach on e-commerce and healthcare datasets; in both cases, we achieve significantly better predictive accuracy as well as managerial insights into the nature of the bias in the proxy data.