# Exposing Blind Spots in Deep Imbalanced Regression Evaluation

arXiv: https://arxiv.org/abs/2609.25152 · HTML: https://mlsanity.com/paper/2609.25152/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 47 references that are in the map.
- First posted on arXiv 21 Sep 2026.

## Builds on (most cited first, 15 of 15)

- [Large-Scale Long-Tailed Recognition in an Open World](https://mlsanity.com/paper/1904.05160/) (2019)
- [Accounting for Variance in Machine Learning Benchmarks](https://mlsanity.com/paper/2103.03098/) (2021)
- [Balanced MSE for Imbalanced Visual Regression](https://mlsanity.com/paper/2203.16427/) (2022)
- [Rank-N-Contrast: Learning Continuous Representations for Regression](https://mlsanity.com/paper/2210.01189/) (2022)
- [Reporting Score Distributions Makes a Difference: Performance Study of LSTM-networks for Sequence Tagging](https://mlsanity.com/paper/1707.09861/) (2017)
- [How Many Random Seeds? Statistical Power Analysis in Deep Reinforcement Learning Experiments](https://mlsanity.com/paper/1806.08295/) (2018)
- [RankSim: Ranking Similarity Regularization for Deep Imbalanced Regression](https://mlsanity.com/paper/2205.15236/) (2022)
- [Improving Deep Regression with Ordinal Entropy](https://mlsanity.com/paper/2301.08915/) (2023)
- [MAESTRO : Adaptive Sparse Attention and Robust Learning for Multimodal Dynamic Time Series](https://mlsanity.com/paper/2509.25278/) (2025)
- [Deep Imbalanced Regression via Hierarchical Classification Adjustment](https://mlsanity.com/paper/2310.17154/) (2023)
- [IM-Context: In-Context Learning for Imbalanced Regression Tasks](https://mlsanity.com/paper/2405.18202/) (2024)
- [Improve Representation for Imbalanced Regression through Geometric Constraints](https://mlsanity.com/paper/2503.00876/) (2025)
- [Uncertainty Voting Ensemble for Imbalanced Deep Regression](https://mlsanity.com/paper/2305.15178/) (2023)
- [Dist Loss: Enhancing Regression in Few-Shot Region through Distribution Distance Constraint](https://mlsanity.com/paper/2411.15216/) (2024)
- [MuViS: Multimodal Virtual Sensing Benchmark](https://mlsanity.com/paper/2603.24602/) (2026)

## Abstract (arXiv)

Deep Imbalanced Regression (DIR) addresses a common failure mode of regression models: target distributions are highly non-uniform, causing models to perform best in densely populated target regions even when reliable performance is required across the full target range. Despite rapid methodological progress, DIR evaluation remains constrained by three blind spots: it is dominated by image-based benchmarks, its standard many-/medium-/few-shot protocol is diagnostic but not decision-complete, and tail-region stability across random seeds has not been systematically evaluated. We revisit DIR evaluation along these three axes. First, we broaden the data domain by evaluating DIR on a multimodal virtual sensing benchmark (textscMuViS) with nine time-series extrinsic regression tasks across six physical domains, where rare target values often correspond to operationally meaningful regimes. Second, we adopt balanced MAE (bMAE) and introduce balanced Mean Absolute Scaled Error (bMASE), a scale-normalized metric for decision-complete comparison across methods and datasets. Third, through a repeated reevaluation of six representative DIR methods across multiple random seeds, we show that the tail regions targeted by DIR exhibit particularly high sensitivity to seed-level variability. Our results show that standard virtual-sensing models exhibit substantial tail degradation hidden by global MAE, that existing DIR methods can improve balanced performance but transfer unevenly to multimodal time-series data, and that tail-region instability remains a largely hidden failure mode under current DIR evaluation practice. Together, these findings and our publicly available code provide a reproducible basis for future DIR research toward regression systems that capture rare target regimes as reliably as common ones.