# Dist Loss: Enhancing Regression in Few-Shot Region through Distribution Distance Constraint

arXiv: https://arxiv.org/abs/2411.15216 · HTML: https://mlsanity.com/paper/2411.15216/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 31 references that are in the map.
- First posted on arXiv 20 Nov 2024.

## Builds on (most cited first, 10 of 10)

- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Large-Scale Long-Tailed Recognition in an Open World](https://mlsanity.com/paper/1904.05160/) (2019)
- [A systematic study of the class imbalance problem in convolutional neural networks](https://mlsanity.com/paper/1710.05381/) (2017)
- [Fast Differentiable Sorting and Ranking](https://mlsanity.com/paper/2002.08871/) (2020)
- [Delving into Deep Imbalanced Regression](https://mlsanity.com/paper/2102.09554/) (2021)
- [Balanced MSE for Imbalanced Visual Regression](https://mlsanity.com/paper/2203.16427/) (2022)
- [Posterior Re-calibration for Imbalanced Datasets](https://mlsanity.com/paper/2010.11820/) (2020)
- [RankSim: Ranking Similarity Regularization for Deep Imbalanced Regression](https://mlsanity.com/paper/2205.15236/) (2022)
- [HOLMES: Health OnLine Model Ensemble Serving for Deep Learning Models in Intensive Care Units](https://mlsanity.com/paper/2008.04063/) (2020)
- [ConR: Contrastive Regularizer for Deep Imbalanced Regression](https://mlsanity.com/paper/2309.06651/) (2023)

## Built on by (most cited first, 3 of 3)

- [More Data or Better Algorithms: Latent Diffusion Augmentation for Deep Imbalanced Regression](https://mlsanity.com/paper/2509.23240/) (2025)
- [Beyond Homoscedasticity: Decoupled Uncertainty Optimization for Deep Imbalanced Regression](https://mlsanity.com/paper/2609.04995/) (2026)
- [Exposing Blind Spots in Deep Imbalanced Regression Evaluation](https://mlsanity.com/paper/2609.25152/) (2026)

## Abstract (arXiv)

Imbalanced data distributions are prevalent in real-world scenarios, posing significant challenges in both imbalanced classification and imbalanced regression tasks. They often cause deep learning models to overfit in areas of high sample density (many-shot regions) while underperforming in areas of low sample density (few-shot regions). This characteristic restricts the utility of deep learning models in various sectors, notably healthcare, where areas with few-shot data hold greater clinical relevance. While recent studies have shown the benefits of incorporating distribution information in imbalanced classification tasks, such strategies are rarely explored in imbalanced regression. In this paper, we address this issue by introducing a novel loss function, termed Dist Loss, designed to minimize the distribution distance between the model's predictions and the target labels in a differentiable manner, effectively integrating distribution information into model training. Dist Loss enables deep learning models to regularize their output distribution during training, effectively enhancing their focus on few-shot regions. We have conducted extensive experiments across three datasets spanning computer vision and healthcare: IMDB-WIKI-DIR, AgeDB-DIR, and ECG-Ka-DIR. The results demonstrate that Dist Loss effectively mitigates the negative impact of imbalanced data distribution on model performance, achieving state-of-the-art results in sparse data regions. Furthermore, Dist Loss is easy to integrate, complementing existing methods.