# Improving Deep Regression with Ordinal Entropy

arXiv: https://arxiv.org/abs/2301.08915 · HTML: https://mlsanity.com/paper/2301.08915/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 31 references that are in the map.
- First posted on arXiv 21 Jan 2023.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [Large-Scale Long-Tailed Recognition in an Open World](https://mlsanity.com/paper/1904.05160/) (2019)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [Opening the Black Box of Deep Neural Networks via Information](https://mlsanity.com/paper/1703.00810/) (2017)
- [Regularizing Neural Networks by Penalizing Confident Output Distributions](https://mlsanity.com/paper/1701.06548/) (2017)
- [Deeper Depth Prediction with Fully Convolutional Residual Networks](https://mlsanity.com/paper/1606.00373/) (2016)
- [From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation](https://mlsanity.com/paper/1907.10326/) (2019)
- [CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes](https://mlsanity.com/paper/1802.10062/) (2018)
- [Distribution-Aware Coordinate Representation for Human Pose Estimation](https://mlsanity.com/paper/1910.06278/) (2019)
- [Delving into Deep Imbalanced Regression](https://mlsanity.com/paper/2102.09554/) (2021)
- [Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation](https://mlsanity.com/paper/1406.2984/) (2014)
- [A unifying mutual information view of metric learning: cross-entropy vs. pairwise losses](https://mlsanity.com/paper/2003.08983/) (2020)
- [Estimating Depth from Monocular Images as Classification Using Deep Fully Convolutional Residual Networks](https://mlsanity.com/paper/1605.02305/) (2016)
- [SimCC: a Simple Coordinate Classification Perspective for Human Pose Estimation](https://mlsanity.com/paper/2107.03332/) (2021)
- [RankSim: Ranking Similarity Regularization for Deep Imbalanced Regression](https://mlsanity.com/paper/2205.15236/) (2022)
- [Discrete-Constrained Regression for Local Counting Models](https://mlsanity.com/paper/2207.09865/) (2022)
- [Class Distance Weighted Cross-Entropy Loss for Ulcerative Colitis Severity Estimation](https://mlsanity.com/paper/2202.05167/) (2022)

## Built on by (most cited first, 30 of 45)

- [Stop Regressing: Training Value Functions via Classification for Scalable Deep RL](https://mlsanity.com/paper/2403.03950/) (2024)
- [Hierarchical Classification Auxiliary Network for Time Series Forecasting](https://mlsanity.com/paper/2405.18975/) (2024)
- [Learning-to-Rank Meets Language: Boosting Language-Driven Ordering Alignment for Ordinal Classification](https://mlsanity.com/paper/2306.13856/) (2023)
- [GraphLand: Evaluating Graph Machine Learning Models on Diverse Industrial Data](https://mlsanity.com/paper/2409.14500/) (2024)
- [Test-time Adaptation for Regression by Subspace Alignment](https://mlsanity.com/paper/2410.03263/) (2024)
- [FGENet: Fine-Grained Extraction Network for Congested Crowd Counting](https://mlsanity.com/paper/2401.01208/) (2024)
- [On the Integration of Spatial-Temporal Knowledge: A Lightweight Approach to Atmospheric Time Series Forecasting](https://mlsanity.com/paper/2408.09695/) (2024)
- [Improve Representation for Imbalanced Regression through Geometric Constraints](https://mlsanity.com/paper/2503.00876/) (2025)
- [Semi-Supervised Crowd Counting with Contextual Modeling: Facilitating Holistic Understanding of Crowd Scenes](https://mlsanity.com/paper/2310.10352/) (2023)
- [Ordinal Classification with Distance Regularization for Robust Brain Age Prediction](https://mlsanity.com/paper/2403.10522/) (2023)
- [Deep Regression Representation Learning with Topology](https://mlsanity.com/paper/2404.13904/) (2024)
- [Is Value Functions Estimation with Classification Plug-and-play for Offline Reinforcement Learning?](https://mlsanity.com/paper/2406.06309/) (2024)
- [Sample Selection via Contrastive Fragmentation for Noisy Label Regression](https://mlsanity.com/paper/2502.17771/) (2025)
- [Rethinking Cell Counting Methods: Decoupling Counting and Localization](https://mlsanity.com/paper/2503.13989/) (2025)
- [SupReMix: Supervised Contrastive Learning for Medical Imaging Regression with Mixup](https://mlsanity.com/paper/2309.16633/) (2023)
- [An Efficient Model-Agnostic Approach for Uncertainty Estimation in Data-Restricted Pedometric Applications](https://mlsanity.com/paper/2409.11985/) (2024)
- [Ordinal-Aware Calibration for Ordinal Classification](https://mlsanity.com/paper/2410.15658/) (2024)
- [AbRank: A Benchmark Dataset and Metric-Learning Framework for Antibody-Antigen Affinity Ranking](https://mlsanity.com/paper/2506.17857/) (2025)
- [LiftFormer: Lifting and Frame Theory Based Monocular Depth Estimation Using Depth and Edge Oriented Subspace Representation](https://mlsanity.com/paper/2604.06576/) (2026)
- [Gradient Aligned Regression via Pairwise Losses](https://mlsanity.com/paper/2402.06104/) (2024)
- [TabTreeFormer: Tabular Data Generation Using Hybrid Tree-Transformer](https://mlsanity.com/paper/2501.01216/) (2025)
- [Improving Deep Regression with Tightness](https://mlsanity.com/paper/2502.09122/) (2025)
- [Towards Depth Foundation Model: Recent Trends in Vision-Based Depth Estimation](https://mlsanity.com/paper/2507.11540/) (2025)
- [Adaptive Random Feature Regularization on Fine-tuning Deep Neural Networks](https://mlsanity.com/paper/2403.10097/) (2024)
- [Leveraging Group Classification with Descending Soft Labeling for Deep Imbalanced Regression](https://mlsanity.com/paper/2412.12327/) (2024)
- [MetricDepth: Enhancing Monocular Depth Estimation with Deep Metric Learning](https://mlsanity.com/paper/2412.20390/) (2024)
- [ACCon: Angle-Compensated Contrastive Regularizer for Deep Regression](https://mlsanity.com/paper/2501.07045/) (2025)
- [Balanced Sharpness-Aware Minimization for Imbalanced Regression](https://mlsanity.com/paper/2508.16973/) (2025)
- [F2Depth: Self-supervised Indoor Monocular Depth Estimation via Optical Flow Consistency and Feature Map Synthesis](https://mlsanity.com/paper/2403.18443/) (2024)
- [Generalized Robust Fundus Photography-based Vision Loss Estimation for High Myopia](https://mlsanity.com/paper/2407.03699/) (2024)

## Abstract (arXiv)

In computer vision, it is often observed that formulating regression problems as a classification task often yields better performance. We investigate this curious phenomenon and provide a derivation to show that classification, with the cross-entropy loss, outperforms regression with a mean squared error loss in its ability to learn high-entropy feature representations. Based on the analysis, we propose an ordinal entropy loss to encourage higher-entropy feature spaces while maintaining ordinal relationships to improve the performance of regression tasks. Experiments on synthetic and real-world regression tasks demonstrate the importance and benefits of increasing entropy for regression.