mlsanityData to 14 Oct 2026

Transfer Learning in Nonparametric Regression with Deep ReLU Networks

arXiv 2608.20255 ↗First posted 20 Aug 2026stat.ML cs.LG stat.MEcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper develops a general transfer learning framework for nonparametric regression with data consisting of multiple groups. Under the assumption that groups share a common structure along with group-specific deviations in additive form, the proposed method employs a two-stage offset learning procedure: the first stage pools data from all groups to estimate an overall mean function, and the second stage estimates offsets for each group, yielding final group-level estimators through additive combination. Upper bounds on the mathcal L2 error are established for the proposed framework, covering a broad class of nonparametric estimators under mild complexity and noise conditions. When instantiated with deep ReLU networks, explicit convergence rates are derived under hierarchical composition models, demonstrating the ability to overcome the curse of dimensionality. Conditions that enable positive transfer with faster rates are considered, including learning with simpler functions and data augmentation through pooling samples across groups. Various simulations and real-data experiments further validate the effectiveness of the proposed method.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 38 of its 71 references that are in the map. First posted on arXiv 20 Aug 2026.

Builds on

38 of 71 references.

References · 38

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  5. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  6. 2017Adversarial Discriminative Domain Adaptationcited by 1,311
  7. 2017VGGFace2: A dataset for recognising faces across pose and agecited by 914
  8. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 433
  9. 2017Age Progression/Regression by Conditional Adversarial Autoencodercited by 429
  10. 2017Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networkscited by 311
  11. 2020Few-Shot Learning via Learning the Representation, Provablycited by 214
  12. 2015The Benefit of Multitask Representation Learningcited by 195
26 more
  1. 2020On the Theory of Transfer Learning: The Importance of Task Diversitycited by 186
  2. 2020Provable Meta-Learning of Linear Representationscited by 161
  3. 2020Covid-19: Automatic detection from X-Ray images utilizing Transfer Learning with Convolutional Neural Networkscited by 92
  4. 2020Transfer Learning for High-dimensional Linear Regression: Prediction, Estimation, and Minimax Optimalitycited by 90
  5. 2019Nonparametric Regression on Low-Dimensional Manifolds using Deep ReLU Networks : Function Approximation and Statistical Recoverycited by 73
  6. 2023How Deep Neural Networks Learn Compositional Data: The Random Hierarchy Modelcited by 71
  7. 2021Transfer Learning under High-dimensional Generalized Linear Modelscited by 70
  8. 2013Adaptive piecewise polynomial estimation via trend filteringcited by 47
  9. 2020Transfer Learning in Large-scale Gaussian Graphical Models with False Discovery Rate Controlcited by 38
  10. 2022Adaptive and Robust Multi-Task Learningcited by 34
  11. 2023Learning from Similar Linear Representations: Adaptivity, Minimaxity, and Robustnesscited by 28
  12. 2021A New Look and Convergence Rate of Federated Multi-Task Learning with Laplacian Regularizationcited by 27
  13. 2016Hypothesis Transfer Learning via Transformation Functionscited by 26
  14. 2020Quantile regression with deep ReLU Networks: Estimators and minimax ratescited by 24
  15. 2020Deep Conditional Transformation Modelscited by 23
  16. 2024Transfer Learning for Nonparametric Regression: Non-asymptotic Minimax Analysis and Adaptive Procedurecited by 22
  17. 2021Representation Learning Beyond Linear Prediction Functionscited by 21
  18. 2025Position: A Theory of Deep Learning Must Include Compositional Sparsitycited by 14
  19. 2024Pretraining and the Lassocited by 8
  20. 2024Smoothness Adaptive Hypothesis Transfer Learningcited by 8
  21. 2013Data enriched linear regressioncited by 7
  22. 2017Additive Models with Trend Filteringcited by 6
  23. 2024Deep Transfer Learning: Model Framework and Error Analysiscited by 5
  24. 2024Dense ReLU Neural Networks for Temporal-spatial Modelcited by 3
  25. 2024Confidence Interval Construction and Conditional Variance Estimation with Dense ReLU Networkscited by 3
  26. 2024Bayesian Transfer Learning for Enhanced Estimation and Inferencecited by 2

Built on by

Not cited yet.