# Multi-Source Transfer Learning of Sparse Single-Index Models

arXiv: https://arxiv.org/abs/2606.29658 · HTML: https://mlsanity.com/paper/2606.29658/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 29 references that are in the map.
- First posted on arXiv 28 Jun 2026.

## Builds on (most cited first, 19 of 19)

- [Spectrally-normalized margin bounds for neural networks](https://mlsanity.com/paper/1706.08498/) (2017)
- [Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data](https://mlsanity.com/paper/2302.07194/) (2023)
- [Diffusion Models are Minimax Optimal Distribution Estimators](https://mlsanity.com/paper/2303.01861/) (2023)
- [Transfer Learning for High-dimensional Linear Regression: Prediction, Estimation, and Minimax Optimality](https://mlsanity.com/paper/2006.10593/) (2020)
- [Transfer Learning under High-dimensional Generalized Linear Models](https://mlsanity.com/paper/2105.14328/) (2021)
- [Gradient Estimators for Implicit Models](https://mlsanity.com/paper/1705.07107/) (2017)
- [A Spectral Approach to Gradient Estimation for Implicit Distributions](https://mlsanity.com/paper/1806.02925/) (2018)
- [Transfer Learning in Large-scale Gaussian Graphical Models with False Discovery Rate Control](https://mlsanity.com/paper/2010.11037/) (2020)
- [Nonparametric Score Estimators](https://mlsanity.com/paper/2005.10099/) (2020)
- [Generalization error bound for denoising score matching under relaxed manifold assumption](https://mlsanity.com/paper/2502.13662/) (2025)
- [Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families](https://mlsanity.com/paper/1506.02564/) (2015)
- [Differentiable Neural Networks with RePU Activation: with Applications to Score Estimation and Isotonic Regression](https://mlsanity.com/paper/2305.00608/) (2023)
- [High-dimensional Varying Index Coefficient Models via Stein's Identity](https://mlsanity.com/paper/1810.07128/) (2018)
- [Tensor Methods for Additive Index Models under Discordance and Heterogeneity](https://mlsanity.com/paper/1807.06693/) (2018)
- [Nonlinear denoising score matching for enhanced learning of structured distributions](https://mlsanity.com/paper/2405.15625/) (2024)
- [Simultaneous Approximation of the Score Function and Its Derivatives by Deep Neural Networks](https://mlsanity.com/paper/2512.23643/) (2025)
- [Profiled Transfer Learning for High Dimensional Linear Model](https://mlsanity.com/paper/2406.00701/) (2024)
- [Transfer Learning for Spatial Autoregressive Models with Application to U.S. Presidential Election Prediction](https://mlsanity.com/paper/2405.15600/) (2024)
- [Nonlinear Multiple Response Regression and Learning of Latent Spaces](https://mlsanity.com/paper/2503.21608/) (2025)

## Built on by (most cited first, 1 of 1)

- [Sufficient Dimesion Reduction via Generalized Stein's Lemma](https://mlsanity.com/paper/2608.15121/) (2026)

## Abstract (arXiv)

Transfer learning leverages knowledge from related source domains to improve learning in a target domain. Recent theoretical advances cover a broad range of regression settings within (generalized) linear models. Despite their diversity, these methods share two common constraints: they assume a known link function or linear structure and require direct access to raw source data. To move beyond these constraints, we propose a source-data-free transfer learning framework based on the single-index model (SIM). Instead of requiring raw source data, our method transfers only summary statistics derived from a generalized Stein's lemma in a one-time communication. This design preserves privacy and avoids side effects caused by dissimilarities of unknown nonlinear link functions across domains. To capture flexible, unknown nonlinearity, we employ a multilayer perceptron guided by the pre-estimated index from the transferred statistics, which significantly mitigates overfitting. Extensive experiments on synthetic data and a real-world application demonstrate consistent improvements over existing (generalized) linear model-based approaches. The proposed framework thus offers a practical, privacy-preserving, and nonlinear-adaptive solution for transfer learning.