# A least distance estimator for a multivariate regression model using deep neural networks

arXiv: https://arxiv.org/abs/2401.03123 · HTML: https://mlsanity.com/paper/2401.03123/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 37 references that are in the map.
- First posted on arXiv 6 Jan 2024.

## Builds on (most cited first, 9 of 9)

- [Efficient Processing of Deep Neural Networks: A Tutorial and Survey](https://mlsanity.com/paper/1703.09039/) (2017)
- [Group Sparse Regularization for Deep Neural Networks](https://mlsanity.com/paper/1607.00485/) (2016)
- [Deep Neural Networks Learn Non-Smooth Functions Effectively](https://mlsanity.com/paper/1802.04474/) (2018)
- [Consistent Feature Selection for Analytic Deep Neural Networks](https://mlsanity.com/paper/2010.08097/) (2020)
- [Nonparametric regression using over-parameterized shallow ReLU neural networks](https://mlsanity.com/paper/2306.08321/) (2023)
- [Automatic Node Selection for Deep Neural Networks using Group Lasso Regularization](https://mlsanity.com/paper/1611.05527/) (2016)
- [Densely connected neural networks for nonlinear regression](https://mlsanity.com/paper/2108.00864/) (2021)
- [Robust Deep Neural Network Estimation for Multi-dimensional Functional Data](https://mlsanity.com/paper/2205.09604/) (2022)
- [Robust nonparametric regression based on deep ReLU neural networks](https://mlsanity.com/paper/2310.20294/) (2023)

## Abstract (arXiv)

We propose a deep neural network (DNN) based least distance (LD) estimator (DNN-LD) for a multivariate regression problem, addressing the limitations of the conventional methods. Due to the flexibility of a DNN structure, both linear and nonlinear conditional mean functions can be easily modeled, and a multivariate regression model can be realized by simply adding extra nodes at the output layer. The proposed method is more efficient in capturing the dependency structure among responses than the least squares loss, and robust to outliers. In addition, we consider L1-type penalization for variable selection, crucial in analyzing high-dimensional data. Namely, we propose what we call (A)GDNN-LD estimator that enjoys variable selection and model estimation simultaneously, by applying the (adaptive) group Lasso penalty to weight parameters in the DNN structure. For the computation, we propose a quadratic smoothing approximation method to facilitate optimizing the non-smooth objective function based on the least distance loss. The simulation studies and a real data analysis demonstrate the promising performance of the proposed method.