# A step towards understanding why classification helps regression

arXiv: https://arxiv.org/abs/2308.10603 · HTML: https://mlsanity.com/paper/2308.10603/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 52 references that are in the map.
- First posted on arXiv 21 Aug 2023.

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [Deep Long-Tailed Learning: A Survey](https://mlsanity.com/paper/2110.04596/) (2021)
- [End-to-End Learning of Geometry and Context for Deep Stereo Regression](https://mlsanity.com/paper/1703.04309/) (2017)
- [3D Bounding Box Estimation Using Deep Learning and Geometry](https://mlsanity.com/paper/1612.00496/) (2016)
- [Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA](https://mlsanity.com/paper/1605.06336/) (2016)
- [A General and Adaptive Robust Loss Function](https://mlsanity.com/paper/1701.03077/) (2017)
- [Delving into Deep Imbalanced Regression](https://mlsanity.com/paper/2102.09554/) (2021)
- [Overcoming Classifier Imbalance for Long-tail Object Detection with Balanced Group Softmax](https://mlsanity.com/paper/2006.10408/) (2020)
- [Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps with Accurate Object Boundaries](https://mlsanity.com/paper/1803.08673/) (2018)
- [Self-supervised Video Representation Learning by Pace Prediction](https://mlsanity.com/paper/2008.05861/) (2020)
- [TCLR: Temporal Contrastive Learning for Video Representation](https://mlsanity.com/paper/2101.07974/) (2021)
- [Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views](https://mlsanity.com/paper/1505.05641/) (2015)
- [Balanced MSE for Imbalanced Visual Regression](https://mlsanity.com/paper/2203.16427/) (2022)
- [Hyperspherical Prototype Networks](https://mlsanity.com/paper/1901.10514/) (2019)
- [Unsupervised learning of action classes with continuous temporal embedding](https://mlsanity.com/paper/1904.04189/) (2019)
- [Designing Deep Networks for Surface Normal Estimation](https://mlsanity.com/paper/1411.4958/) (2014)
- [Viewpoints and Keypoints](https://mlsanity.com/paper/1411.6067/) (2014)
- [RankSim: Ranking Similarity Regularization for Deep Imbalanced Regression](https://mlsanity.com/paper/2205.15236/) (2022)
- [A Comprehensive Analysis of Deep Regression](https://mlsanity.com/paper/1803.08450/) (2018)
- [Pose from Shape: Deep Pose Estimation for Arbitrary 3D Objects](https://mlsanity.com/paper/1906.05105/) (2019)
- [Dense Optical Flow Prediction from a Static Image](https://mlsanity.com/paper/1505.00295/) (2015)
- [Horizon Lines in the Wild](https://mlsanity.com/paper/1604.02129/) (2016)
- [Robust Optimization for Deep Regression](https://mlsanity.com/paper/1505.06606/) (2015)

## Built on by (most cited first, 15 of 15)

- [Stop Regressing: Training Value Functions via Classification for Scalable Deep RL](https://mlsanity.com/paper/2403.03950/) (2024)
- [Hierarchical Classification Auxiliary Network for Time Series Forecasting](https://mlsanity.com/paper/2405.18975/) (2024)
- [FoldToken: Learning Protein Language via Vector Quantization and Beyond](https://mlsanity.com/paper/2403.09673/) (2024)
- [An Efficient Model-Agnostic Approach for Uncertainty Estimation in Data-Restricted Pedometric Applications](https://mlsanity.com/paper/2409.11985/) (2024)
- [Learning Relative Gene Expression Trends from Pathology Images in Spatial Transcriptomics](https://mlsanity.com/paper/2512.06612/) (2025)
- [V-Hands: Touchscreen-based Hand Tracking for Remote Whiteboard Interaction](https://mlsanity.com/paper/2409.13347/) (2024)
- [Improving Deep Regression with Tightness](https://mlsanity.com/paper/2502.09122/) (2025)
- [Structural Entropy Guided Probabilistic Coding](https://mlsanity.com/paper/2412.08841/) (2024)
- [Balanced Sharpness-Aware Minimization for Imbalanced Regression](https://mlsanity.com/paper/2508.16973/) (2025)
- [CardiacCLIP: Video-based CLIP Adaptation for LVEF Prediction in a Few-shot Manner](https://mlsanity.com/paper/2509.17065/) (2025)
- [A Latent Risk-Aware Machine Learning Approach for Predicting Operational Success in Clinical Trials based on TrialsBank](https://mlsanity.com/paper/2603.29041/) (2026)
- [Rare-Aware Autoencoding: Reconstructing Spatially Imbalanced Data](https://mlsanity.com/paper/2604.02031/) (2026)
- [How Should Transformers Encode Numeric Values in Electronic Health Records?](https://mlsanity.com/paper/2607.01391/) (2026)
- [Error Aware Distribution Prediction for Lightweight Implicit Neural Representations](https://mlsanity.com/paper/2607.10068/) (2026)
- [PhyRestore: Physics-Structured Latent-Factor Restoration](https://mlsanity.com/paper/2609.19776/) (2026)

## Abstract (arXiv)

A number of computer vision deep regression approaches report improved results when adding a classification loss to the regression loss. Here, we explore why this is useful in practice and when it is beneficial. To do so, we start from precisely controlled dataset variations and data samplings and find that the effect of adding a classification loss is the most pronounced for regression with imbalanced data. We explain these empirical findings by formalizing the relation between the balanced and imbalanced regression losses. Finally, we show that our findings hold on two real imbalanced image datasets for depth estimation (NYUD2-DIR), and age estimation (IMDB-WIKI-DIR), and on the problem of imbalanced video progress prediction (Breakfast). Our main takeaway is: for a regression task, if the data sampling is imbalanced, then add a classification loss.