# Adaptive deep nonparametric regression from dependent data under covariate shift

arXiv: https://arxiv.org/abs/2607.20309 · HTML: https://mlsanity.com/paper/2607.20309/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 50 references that are in the map.
- First posted on arXiv 22 Jul 2026.

## Builds on (most cited first, 26 of 26)

- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Measuring Robustness to Natural Distribution Shifts in Image Classification](https://mlsanity.com/paper/2007.00644/) (2020)
- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [Domain Adaptation for Medical Image Analysis: A Survey](https://mlsanity.com/paper/2102.09508/) (2021)
- [Optimal approximation of piecewise smooth functions using deep ReLU neural networks](https://mlsanity.com/paper/1709.05289/) (2017)
- [Deep Neural Networks Learn Non-Smooth Functions Effectively](https://mlsanity.com/paper/1802.04474/) (2018)
- [Optimally tackling covariate shift in RKHS-based nonparametric regression](https://mlsanity.com/paper/2205.02986/) (2022)
- [Fast convergence rates of deep neural networks for classification](https://mlsanity.com/paper/1812.03599/) (2018)
- [Smooth function approximation by deep neural networks with general activation functions](https://mlsanity.com/paper/1906.06903/) (2019)
- [How do noise tails impact on deep ReLU networks?](https://mlsanity.com/paper/2203.10418/) (2022)
- [Quantile regression with deep ReLU Networks: Estimators and minimax rates](https://mlsanity.com/paper/2010.08236/) (2020)
- [Nonconvex sparse regularization for deep neural networks and its optimality](https://mlsanity.com/paper/2003.11769/) (2020)
- [Advantage of Deep Neural Networks for Estimating Functions with Singularity on Hypersurfaces](https://mlsanity.com/paper/2011.02256/) (2020)
- [Theoretical analysis of deep neural networks for temporally dependent observations](https://mlsanity.com/paper/2210.11530/) (2022)
- [On the rate of convergence of a deep recurrent neural network estimate in a regression problem with dependent data](https://mlsanity.com/paper/2011.00328/) (2020)
- [Towards a Unified Analysis of Kernel-based Methods Under Covariate Shift](https://mlsanity.com/paper/2310.08237/) (2023)
- [Adaptive deep learning for nonlinear time series models](https://mlsanity.com/paper/2207.02546/) (2022)
- [Excess risk bound for deep learning under weak dependence](https://mlsanity.com/paper/2302.07503/) (2023)
- [Learning theory estimates with observations from general stationary stochastic processes](https://mlsanity.com/paper/1605.02887/) (2016)
- [Robust deep learning from weakly dependent data](https://mlsanity.com/paper/2405.05081/) (2024)
- [Deep Transfer Learning: Model Framework and Error Analysis](https://mlsanity.com/paper/2410.09383/) (2024)
- [Optimal Kernel Quantile Learning with Random Features](https://mlsanity.com/paper/2408.13591/) (2024)
- [Minimax optimality of deep neural networks on dependent data via PAC-Bayes bounds](https://mlsanity.com/paper/2410.21702/) (2024)
- [Deep learning from strongly mixing observations: Sparse-penalized regularization and minimax optimality](https://mlsanity.com/paper/2406.08321/) (2024)
- [A general framework for deep learning](https://mlsanity.com/paper/2512.23425/) (2025)
- [Deep Regression for Repeated Measurements under Covariate Shift](https://mlsanity.com/paper/2605.24854/) (2026)

## Abstract (arXiv)

Covariate shift often occurs because, in many real applications, the source and the target observations may be generated from different distributions. In this case, the standard metric under the source distribution is not appropriate. This paper considers deep neural network estimators for nonparametric quantile and Huber regression under covariate shift and from dependent observations. We deal with a generalized Bernstein-type inequality that is satisfied by many classical models, including i.i.d. observations, φ-mixing, strong mixing, and C-mixing processes. To perform the covariate shift phenomenon, we propose a sparse-penalized deep neural network (SPDNN) estimator that takes into account the discrepancy between the source and target distributions of the data. When the density ratio (between the source and target distributions of the covariate) is unknown, a two steps pre-training procedure is carried out: the first step is devoted to the construction of a least squares SPDNN estimator of the density ratio; which is used in the second step to perform a pre-training reweighted SPDNN estimator of the regression function. For both the quantile and the Huber regression, non-asymptotic error bounds of the proposed SPDNN estimators are established in the class of H"older smooth functions. These estimators can adaptively attain (up to a logarithmic factor) the minimax optimal convergence rate from i.i.d. data as well as from several classical time series models.