# A Comprehensive Analysis of Deep Regression

arXiv: https://arxiv.org/abs/1803.08450 · HTML: https://mlsanity.com/paper/1803.08450/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Mar 2018.

## Built on by (most cited first, 30 of 48)

- [Sequential Predictive Conformal Inference for Time Series](https://mlsanity.com/paper/2212.03463/) (2022)
- [A Review on Deep Learning in UAV Remote Sensing](https://mlsanity.com/paper/2101.10861/) (2021)
- [Principal Uncertainty Quantification with Spatial Correlation for Image Restoration Problems](https://mlsanity.com/paper/2305.10124/) (2023)
- [A step towards understanding why classification helps regression](https://mlsanity.com/paper/2308.10603/) (2023)
- [How Reliable is Your Regression Model's Uncertainty Under Real-World Distribution Shifts?](https://mlsanity.com/paper/2302.03679/) (2023)
- [Diffusion MRI with Machine Learning](https://mlsanity.com/paper/2402.00019/) (2024)
- [Test-time Adaptation for Regression by Subspace Alignment](https://mlsanity.com/paper/2410.03263/) (2024)
- [On the benefits of maximum likelihood estimation for Regression and Forecasting](https://mlsanity.com/paper/2106.10370/) (2021)
- [DPNNet-2.0 Part I: Finding hidden planets from simulated images of protoplanetary disk gaps](https://mlsanity.com/paper/2107.09086/) (2021)
- [Backdoor Attack and Defense for Deep Regression](https://mlsanity.com/paper/2109.02381/) (2021)
- [AI/ML Algorithms and Applications in VLSI Design and Technology](https://mlsanity.com/paper/2202.10015/) (2022)
- [Few-shot calibration of low-cost air pollution (PM2.5) sensors using meta-learning](https://mlsanity.com/paper/2108.00640/) (2021)
- [Densely connected neural networks for nonlinear regression](https://mlsanity.com/paper/2108.00864/) (2021)
- [Symbolic Regression for Space Applications: Differentiable Cartesian Genetic Programming Powered by Multi-objective Memetic Algorithms](https://mlsanity.com/paper/2206.06213/) (2022)
- [An Additive Autoencoder for Dimension Estimation](https://mlsanity.com/paper/2210.06773/) (2022)
- [Contrastive Learning with Adaptive Neighborhoods for Brain Age Prediction on 3D Stiffness Maps](https://mlsanity.com/paper/2408.00527/) (2024)
- [Exploiting Memory-aware Q-distribution Prediction for Nuclear Fusion via Modern Hopfield Network](https://mlsanity.com/paper/2410.08889/) (2024)
- [Defending Deep Regression Models against Backdoor Attacks](https://mlsanity.com/paper/2411.04811/) (2024)
- [Real-Time Optimal Trajectory Planning for Autonomous Vehicles and Lap Time Simulation Using Machine Learning](https://mlsanity.com/paper/2102.02315/) (2021)
- [A Neural Anthropometer Learning from Body Dimensions Computed on Human 3D Meshes](https://mlsanity.com/paper/2110.04064/) (2021)
- [Probabilistic Regression with Huber Distributions](https://mlsanity.com/paper/2111.10296/) (2021)
- [Distributional loss for convolutional neural network regression and application to GNSS multi-path estimation](https://mlsanity.com/paper/2206.01473/) (2022)
- [Passive Heart Rate Monitoring During Smartphone Use in Everyday Life](https://mlsanity.com/paper/2503.03783/) (2025)
- [Uncertainty Quantification for Deep Regression using Contextualised Normalizing Flows](https://mlsanity.com/paper/2512.00835/) (2025)
- [Overestimation learning with guarantees](https://mlsanity.com/paper/2101.11717/) (2021)
- [Using Robust Regression to Find Font Usage Trends](https://mlsanity.com/paper/2106.15232/) (2021)
- [Robust and Active Learning for Deep Neural Network Regression](https://mlsanity.com/paper/2107.13124/) (2021)
- [A Simple Approach to Image Tilt Correction with Self-Attention MobileNet for Smartphones](https://mlsanity.com/paper/2111.00398/) (2021)
- [Survey of Deep Learning Methods for Inverse Problems](https://mlsanity.com/paper/2111.04731/) (2021)
- [ECG-Based Electrolyte Prediction: Evaluating Regression and Probabilistic Methods](https://mlsanity.com/paper/2212.13890/) (2022)

## Abstract (arXiv)

Deep learning revolutionized data science, and recently its popularity has grown exponentially, as did the amount of papers employing deep networks. Vision tasks, such as human pose estimation, did not escape from this trend. There is a large number of deep models, where small changes in the network architecture, or in the data pre-processing, together with the stochastic nature of the optimization procedures, produce notably different results, making extremely difficult to sift methods that significantly outperform others. This situation motivates the current study, in which we perform a systematic evaluation and statistical analysis of vanilla deep regression, i.e. convolutional neural networks with a linear regression top layer. This is the first comprehensive analysis of deep regression techniques. We perform experiments on four vision problems, and report confidence intervals for the median performance as well as the statistical significance of the results, if any. Surprisingly, the variability due to different data pre-processing procedures generally eclipses the variability due to modifications in the network architecture. Our results reinforce the hypothesis according to which, in general, a general-purpose network (e.g. VGG-16 or ResNet-50) adequately tuned can yield results close to the state-of-the-art without having to resort to more complex and ad-hoc regression models.