# Robust Optimization for Deep Regression

arXiv: https://arxiv.org/abs/1505.06606 · HTML: https://mlsanity.com/paper/1505.06606/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 May 2015.

## Built on by (most cited first, 22 of 22)

- [Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression](https://mlsanity.com/paper/2104.02300/) (2021)
- [Human Pose as Compositional Tokens](https://mlsanity.com/paper/2303.11638/) (2023)
- [Regression as Classification: Influence of Task Formulation on Neural Network Features](https://mlsanity.com/paper/2211.05641/) (2022)
- [NeuroGen: activation optimized image synthesis for discovery neuroscience](https://mlsanity.com/paper/2105.07140/) (2021)
- [Deep Learning for Camera Calibration and Beyond: A Survey](https://mlsanity.com/paper/2303.10559/) (2023)
- [Adaptive Contrast for Image Regression in Computer-Aided Disease Assessment](https://mlsanity.com/paper/2112.11700/) (2021)
- [A step towards understanding why classification helps regression](https://mlsanity.com/paper/2308.10603/) (2023)
- [Unsharp Mask Guided Filtering](https://mlsanity.com/paper/2106.01428/) (2021)
- [Injecting Domain Knowledge from Empirical Interatomic Potentials to Neural Networks for Predicting Material Properties](https://mlsanity.com/paper/2210.08047/) (2022)
- [DeepMoM: Robust Deep Learning With Median-of-Means](https://mlsanity.com/paper/2105.14035/) (2021)
- [Bandwidth-based Step-Sizes for Non-Convex Stochastic Optimization](https://mlsanity.com/paper/2106.02888/) (2021)
- [Building Bridges between Regression, Clustering, and Classification](https://mlsanity.com/paper/2502.02996/) (2025)
- [Using Robust Regression to Find Font Usage Trends](https://mlsanity.com/paper/2106.15232/) (2021)
- [Attention-based Vehicle Self-Localization with HD Feature Maps](https://mlsanity.com/paper/2107.07787/) (2021)
- [Global quantitative robustness of regression feed-forward neural networks](https://mlsanity.com/paper/2211.10124/) (2022)
- [Hierarchical Multiresolution Feature- and Prior-based Graphs for Classification](https://mlsanity.com/paper/2306.02143/) (2023)
- [Direct Unsupervised Denoising](https://mlsanity.com/paper/2310.18116/) (2023)
- [Cross-Domain Knowledge Distillation for Low-Resolution Human Pose Estimation](https://mlsanity.com/paper/2405.11448/) (2024)
- [Focus on Low-Resolution Information: Multi-Granular Information-Lossless Model for Low-Resolution Human Pose Estimation](https://mlsanity.com/paper/2405.12247/) (2024)
- [LEARN: An Invex Loss for Outlier Oblivious Robust Online Optimization](https://mlsanity.com/paper/2408.06297/) (2024)
- [What should an AI assessor optimise for?](https://mlsanity.com/paper/2502.00365/) (2025)
- [Heart rate and respiratory rate prediction from noisy real-world smartphone based on Deep Learning methods](https://mlsanity.com/paper/2506.22460/) (2025)

## Abstract (arXiv)

Convolutional Neural Networks (ConvNets) have successfully contributed to improve the accuracy of regression-based methods for computer vision tasks such as human pose estimation, landmark localization, and object detection. The network optimization has been usually performed with L2 loss and without considering the impact of outliers on the training process, where an outlier in this context is defined by a sample estimation that lies at an abnormal distance from the other training sample estimations in the objective space. In this work, we propose a regression model with ConvNets that achieves robustness to such outliers by minimizing Tukey's biweight function, an M-estimator robust to outliers, as the loss function for the ConvNet. In addition to the robust loss, we introduce a coarse-to-fine model, which processes input images of progressively higher resolutions for improving the accuracy of the regressed values. In our experiments, we demonstrate faster convergence and better generalization of our robust loss function for the tasks of human pose estimation and age estimation from face images. We also show that the combination of the robust loss function with the coarse-to-fine model produces comparable or better results than current state-of-the-art approaches in four publicly available human pose estimation datasets.