# Representation Retrieval Learning for Heterogeneous Data Integration

arXiv: https://arxiv.org/abs/2503.09494 · HTML: https://mlsanity.com/paper/2503.09494/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 86 references that are in the map.
- First posted on arXiv 12 Mar 2025.

## Builds on (most cited first, 30 of 40)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Spectrally-normalized margin bounds for neural networks](https://mlsanity.com/paper/1706.08498/) (2017)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [Learning Structured Sparsity in Deep Neural Networks](https://mlsanity.com/paper/1608.03665/) (2016)
- [A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks](https://mlsanity.com/paper/1707.09564/) (2017)
- [Size-Independent Sample Complexity of Neural Networks](https://mlsanity.com/paper/1712.06541/) (2017)
- [Few-Shot Learning via Learning the Representation, Provably](https://mlsanity.com/paper/2002.09434/) (2020)
- [Provable Meta-Learning of Linear Representations](https://mlsanity.com/paper/2002.11684/) (2020)
- [Transfer Learning for High-dimensional Linear Regression: Prediction, Estimation, and Minimax Optimality](https://mlsanity.com/paper/2006.10593/) (2020)
- [LassoNet: A Neural Network with Feature Sparsity](https://mlsanity.com/paper/1907.12207/) (2019)
- [Transfer Learning under High-dimensional Generalized Linear Models](https://mlsanity.com/paper/2105.14328/) (2021)
- [Causal inference methods for combining randomized trials and observational studies: a review](https://mlsanity.com/paper/2011.08047/) (2020)
- [Sparse-Input Neural Networks for High-dimensional Nonparametric Regression and Classification](https://mlsanity.com/paper/1711.07592/) (2017)
- [Adaptive and Robust Multi-Task Learning](https://mlsanity.com/paper/2202.05250/) (2022)
- [Factor Augmented Sparse Throughput Deep ReLU Neural Networks for High Dimensional Regression](https://mlsanity.com/paper/2210.02002/) (2022)
- [Angle-Based Joint and Individual Variation Explained](https://mlsanity.com/paper/1704.02060/) (2017)
- [Combining multiple observational data sources to estimate causal effects](https://mlsanity.com/paper/1801.00802/) (2018)
- [Learning from Similar Linear Representations: Adaptivity, Minimaxity, and Robustness](https://mlsanity.com/paper/2303.17765/) (2023)
- [Structural Learning and Integrative Decomposition of Multi-View Data](https://mlsanity.com/paper/1707.06573/) (2017)
- [Structured Matrix Completion with Applications to Genomic Data Integration](https://mlsanity.com/paper/1504.01823/) (2015)
- [Robust angle-based transfer learning in high dimensions](https://mlsanity.com/paper/2210.12759/) (2022)
- [Group Factor Analysis](https://mlsanity.com/paper/1411.5799/) (2014)
- [Exploring and Exploiting Uncertainty for Incomplete Multi-View Classification](https://mlsanity.com/paper/2304.05165/) (2023)
- [Integrating multi-source block-wise missing data in model selection](https://mlsanity.com/paper/1901.03797/) (2019)
- [Data Integration in Causal Inference](https://mlsanity.com/paper/2110.01106/) (2021)
- [Multi-source Learning via Completion of Block-wise Overlapping Noisy Matrices](https://mlsanity.com/paper/2105.10360/) (2021)
- [Pretraining and the Lasso](https://mlsanity.com/paper/2401.12911/) (2024)
- [Integrated Principal Components Analysis](https://mlsanity.com/paper/1810.00832/) (2018)

## Built on by (most cited first, 5 of 5)

- [Multi-Task Learning with Covariate-Overlap Regularization](https://mlsanity.com/paper/2505.24281/) (2025)
- [Syntax-Guided Diffusion Language Models with User-Integrated Personalization](https://mlsanity.com/paper/2510.01028/) (2025)
- [Statistical Learning for Latent Embedding Alignment with Application to Brain Encoding and Decoding](https://mlsanity.com/paper/2603.21042/) (2026)
- [Identifiable Multimodal Causal Representation Learning under Partial Latent Sharing](https://mlsanity.com/paper/2605.19135/) (2026)
- [Hierarchical Projection for Adaptive Knowledge Transfer](https://mlsanity.com/paper/2606.08691/) (2026)

## Abstract (arXiv)

In the era of big data, large-scale, multi-source, multi-modality datasets are increasingly ubiquitous, offering unprecedented opportunities for predictive modeling and scientific discovery. However, these datasets often exhibit complex heterogeneity, such as covariates shift, posterior drift, and blockwise missingness, which worsen predictive performance of existing supervised learning algorithms. To address these challenges simultaneously, we propose a novel Representation Retrieval (R2) framework, which integrates a dictionary of representation learning modules (representer dictionary) with data source-specific sparsity-induced machine learning model (learners). Under the R2 framework, we introduce the notion of integrativeness for each representer, and propose a novel Selective Integration Penalty (SIP) to explicitly encourage more integrative representers to improve predictive performance. Theoretically, we show that the excess risk bound of the R2 framework is characterized by the integrativeness of representers, and SIP effectively improves the excess risk. Extensive simulation studies validate the superior performance of R2 framework and the effect of SIP. We further apply our method to two real-world datasets to confirm its empirical success.