mlsanityData to 14 Oct 2026

Adaptive deep nonparametric regression from dependent data under covariate shift

arXiv 2607.20309 ↗First posted 22 Jul 2026stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Covariate shift often occurs because, in many real applications, the source and the target observations may be generated from different distributions. In this case, the standard metric under the source distribution is not appropriate. This paper considers deep neural network estimators for nonparametric quantile and Huber regression under covariate shift and from dependent observations. We deal with a generalized Bernstein-type inequality that is satisfied by many classical models, including i.i.d. observations, φ-mixing, strong mixing, and C-mixing processes. To perform the covariate shift phenomenon, we propose a sparse-penalized deep neural network (SPDNN) estimator that takes into account the discrepancy between the source and target distributions of the data. When the density ratio (between the source and target distributions of the covariate) is unknown, a two steps pre-training procedure is carried out: the first step is devoted to the construction of a least squares SPDNN estimator of the density ratio; which is used in the second step to perform a pre-training reweighted SPDNN estimator of the regression function. For both the quantile and the Huber regression, non-asymptotic error bounds of the proposed SPDNN estimators are established in the class of H"older smooth functions. These estimators can adaptively attain (up to a logarithmic factor) the minimax optimal convergence rate from i.i.d. data as well as from several classical time series models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 50 references that are in the map. First posted on arXiv 22 Jul 2026.

Builds on

26 of 50 references.

References · 26

  1. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,375
  2. 2020Measuring Robustness to Natural Distribution Shifts in Image Classificationcited by 454
  3. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 433
  4. 2021Domain Adaptation for Medical Image Analysis: A Surveycited by 352
  5. 2017Optimal approximation of piecewise smooth functions using deep ReLU neural networkscited by 179
  6. 2018Deep Neural Networks Learn Non-Smooth Functions Effectivelycited by 57
  7. 2022Optimally tackling covariate shift in RKHS-based nonparametric regressioncited by 54
  8. 2018Fast convergence rates of deep neural networks for classificationcited by 46
  9. 2019Smooth function approximation by deep neural networks with general activation functionscited by 30
  10. 2022How do noise tails impact on deep ReLU networks?cited by 25
  11. 2020Quantile regression with deep ReLU Networks: Estimators and minimax ratescited by 24
  12. 2020Nonconvex sparse regularization for deep neural networks and its optimalitycited by 21
14 more
  1. 2020Advantage of Deep Neural Networks for Estimating Functions with Singularity on Hypersurfacescited by 17
  2. 2022Theoretical analysis of deep neural networks for temporally dependent observationscited by 16
  3. 2020On the rate of convergence of a deep recurrent neural network estimate in a regression problem with dependent datacited by 14
  4. 2023Towards a Unified Analysis of Kernel-based Methods Under Covariate Shiftcited by 13
  5. 2022Adaptive deep learning for nonlinear time series modelscited by 11
  6. 2023Excess risk bound for deep learning under weak dependencecited by 9
  7. 2016Learning theory estimates with observations from general stationary stochastic processescited by 7
  8. 2024Robust deep learning from weakly dependent datacited by 5
  9. 2024Deep Transfer Learning: Model Framework and Error Analysiscited by 5
  10. 2024Optimal Kernel Quantile Learning with Random Featurescited by 4
  11. 2024Minimax optimality of deep neural networks on dependent data via PAC-Bayes boundscited by 4
  12. 2024Deep learning from strongly mixing observations: Sparse-penalized regularization and minimax optimalitycited by 3
  13. 2025A general framework for deep learningcited by 2
  14. 2026Deep Regression for Repeated Measurements under Covariate Shiftcited by 1

Built on by

Not cited yet.