# Improving Deep Regression with Tightness

arXiv: https://arxiv.org/abs/2502.09122 · HTML: https://mlsanity.com/paper/2502.09122/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 27 references that are in the map.
- First posted on arXiv 13 Feb 2025.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [Opening the Black Box of Deep Neural Networks via Information](https://mlsanity.com/paper/1703.00810/) (2017)
- [Delving into Deep Imbalanced Regression](https://mlsanity.com/paper/2102.09554/) (2021)
- [Stop Regressing: Training Value Functions via Classification for Scalable Deep RL](https://mlsanity.com/paper/2403.03950/) (2024)
- [Information Dropout: Learning Optimal Representations Through Noisy Computation](https://mlsanity.com/paper/1611.01353/) (2016)
- [Topological Autoencoders](https://mlsanity.com/paper/1906.00722/) (2019)
- [A unifying mutual information view of metric learning: cross-entropy vs. pairwise losses](https://mlsanity.com/paper/2003.08983/) (2020)
- [How Does Information Bottleneck Help Deep Learning?](https://mlsanity.com/paper/2305.18887/) (2023)
- [Intrinsic Dimension, Persistent Homology and Generalization in Neural Networks](https://mlsanity.com/paper/2111.13171/) (2021)
- [SimCC: a Simple Coordinate Classification Perspective for Human Pose Estimation](https://mlsanity.com/paper/2107.03332/) (2021)
- [RankSim: Ranking Similarity Regularization for Deep Imbalanced Regression](https://mlsanity.com/paper/2205.15236/) (2022)
- [Improving Deep Regression with Ordinal Entropy](https://mlsanity.com/paper/2301.08915/) (2023)
- [Discrete-Constrained Regression for Local Counting Models](https://mlsanity.com/paper/2207.09865/) (2022)
- [ConR: Contrastive Regularizer for Deep Imbalanced Regression](https://mlsanity.com/paper/2309.06651/) (2023)
- [Doubly-Stochastic Normalization of the Gaussian Kernel is Robust to Heteroskedastic Noise](https://mlsanity.com/paper/2006.00402/) (2020)
- [A step towards understanding why classification helps regression](https://mlsanity.com/paper/2308.10603/) (2023)
- [Deep Regression Representation Learning with Topology](https://mlsanity.com/paper/2404.13904/) (2024)

## Built on by (most cited first, 2 of 2)

- [Your Data Manifold is Secretly a Reward Model: Shell-LCC for Text-to-Video Generation](https://mlsanity.com/paper/2606.30248/) (2026)
- [Revisiting Tabular Deep Regression: Objectives and the Limits of Global Early Stopping](https://mlsanity.com/paper/2610.13909/) (2026)

## Abstract (arXiv)

For deep regression, preserving the ordinality of the targets with respect to the feature representation improves performance across various tasks. However, a theoretical explanation for the benefits of ordinality is still lacking. This work reveals that preserving ordinality reduces the conditional entropy H(Z|Y) of representation Z conditional on the target Y. However, our findings reveal that typical regression losses do little to reduce H(Z|Y), even though it is vital for generalization performance. With this motivation, we introduce an optimal transport-based regularizer to preserve the similarity relationships of targets in the feature space to reduce H(Z|Y). Additionally, we introduce a simple yet efficient strategy of duplicating the regressor targets, also with the aim of reducing H(Z|Y). Experiments on three real-world regression tasks verify the effectiveness of our strategies to improve deep regression. Code: https://github.com/needylove/Regressiontightness.