# Nonparametric regression with modified ReLU networks

arXiv: https://arxiv.org/abs/2207.08306 · HTML: https://mlsanity.com/paper/2207.08306/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 20 references that are in the map.
- First posted on arXiv 17 Jul 2022.

## Builds on (most cited first, 9 of 9)

- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality](https://mlsanity.com/paper/1810.08033/) (2018)
- [Statistical Guarantees for Regularized Neural Networks](https://mlsanity.com/paper/2006.00294/) (2020)
- [Nonconvex sparse regularization for deep neural networks and its optimality](https://mlsanity.com/paper/2003.11769/) (2020)
- [On Deep Instrumental Variables Estimate](https://mlsanity.com/paper/2004.14954/) (2020)
- [Optimal Nonparametric Inference via Deep Neural Network](https://mlsanity.com/paper/1902.01687/) (2019)
- [Estimation of the Mean Function of Functional Data via Deep Neural Networks](https://mlsanity.com/paper/2012.04573/) (2020)
- [Calibrating multi-dimensional complex ODE from noisy data via deep neural networks](https://mlsanity.com/paper/2106.03591/) (2021)
- [Fast learning rate of deep learning via a kernel perspective](https://mlsanity.com/paper/1705.10182/) (2017)

## Abstract (arXiv)

We consider regression estimation with modified ReLU neural networks in which network weight matrices are first modified by a function α before being multiplied by input vectors. We give an example of continuous, piecewise linear function α for which the empirical risk minimizers over the classes of modified ReLU networks with l1 and squared l2 penalties attain, up to a logarithmic factor, the minimax rate of prediction of unknown β-smooth function.