# Revisiting Tabular Deep Regression: Objectives and the Limits of Global Early Stopping

arXiv: https://arxiv.org/abs/2610.13909 · HTML: https://mlsanity.com/paper/2610.13909/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 44 references that are in the map.
- First posted on arXiv 9 Oct 2026.

## Builds on (most cited first, 30 of 30)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [On Embeddings for Numerical Features in Tabular Deep Learning](https://mlsanity.com/paper/2203.05556/) (2022)
- [Stop Regressing: Training Value Functions via Classification for Scalable Deep RL](https://mlsanity.com/paper/2403.03950/) (2024)
- [TabICLv2: A better, faster, scalable, and open tabular foundation model](https://mlsanity.com/paper/2602.11139/) (2026)
- [On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networks](https://mlsanity.com/paper/2203.09168/) (2022)
- [TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensembling](https://mlsanity.com/paper/2410.24210/) (2024)
- [CARD: Classification and Regression Diffusion Models](https://mlsanity.com/paper/2206.07275/) (2022)
- [Better by Default: Strong Pre-Tuned MLPs and Boosted Trees on Tabular Data](https://mlsanity.com/paper/2407.04491/) (2024)
- [Rank-N-Contrast: Learning Continuous Representations for Regression](https://mlsanity.com/paper/2210.01189/) (2022)
- [Improving Regression Performance with Distributional Losses](https://mlsanity.com/paper/1806.04613/) (2018)
- [Improving Deep Regression with Ordinal Entropy](https://mlsanity.com/paper/2301.08915/) (2023)
- [Early Stopping in Deep Networks: Double Descent and How to Eliminate it](https://mlsanity.com/paper/2007.10099/) (2020)
- [Faithful Heteroscedastic Regression with Neural Networks](https://mlsanity.com/paper/2212.09184/) (2022)
- [Fast calibrated additive quantile regression](https://mlsanity.com/paper/1707.03307/) (2017)
- [Beyond IID: How General Are Tabular Foundation Models, Really?](https://mlsanity.com/paper/2606.30410/) (2026)
- [Rethinking Early Stopping: Refine, Then Calibrate](https://mlsanity.com/paper/2501.19195/) (2025)
- [High-Dimensional Quantile Regression: Convolution Smoothing and Concave Regularization](https://mlsanity.com/paper/2109.05640/) (2021)
- [Decoding-based Regression](https://mlsanity.com/paper/2501.19383/) (2025)
- [Adaptive Label Smoothing](https://mlsanity.com/paper/2009.06432/) (2020)
- [Understanding Pathologies of Deep Heteroskedastic Regression](https://mlsanity.com/paper/2306.16717/) (2023)
- [Benchmarking Optimizers for MLPs in Tabular Deep Learning](https://mlsanity.com/paper/2604.15297/) (2026)
- [Token-Level Fitting Issues of Seq2seq Models](https://mlsanity.com/paper/2305.04493/) (2023)
- [A Robust Quantile Huber Loss With Interpretable Parameter Adjustment In Distributional Reinforcement Learning](https://mlsanity.com/paper/2401.02325/) (2024)
- [Gradient Aligned Regression via Pairwise Losses](https://mlsanity.com/paper/2402.06104/) (2024)
- [Building Bridges between Regression, Clustering, and Classification](https://mlsanity.com/paper/2502.02996/) (2025)
- [Improving Deep Regression with Tightness](https://mlsanity.com/paper/2502.09122/) (2025)
- [Unveiling the Role of Data Uncertainty in Tabular Deep Learning](https://mlsanity.com/paper/2509.04430/) (2025)
- [TabPack: Efficient Hyperparameter Ensembles for Tabular Deep Learning](https://mlsanity.com/paper/2607.05380/) (2026)

## Abstract (arXiv)

While deep learning models for tabular data have recently received significant research attention, the choice of objective functions for regression tasks remains an understudied degree of freedom. Several lines of research offer alternatives to the standard Mean Squared Error (MSE), but differences in datasets, backbones, and evaluation protocols make their practical benefits difficult to compare. In this work, we address this gap by performing a systematic re-evaluation of regression objectives across a broad collection of established tabular datasets and multiple neural backbones. Our comparison reveals that many proposed objectives provide limited or inconsistent gains over a properly tuned MSE baseline, while quantile loss yields consistent aggregate improvements over MSE across the tested backbones. We then identify a limitation that remains even with this strong objective: regions with different levels of target noise can reach their best predictive performance at different training epochs. Consequently, a single global early-stopping checkpoint can impose a compromise between uncertainty regions, even when the objective can represent heteroskedastic target distributions. Motivated by this limitation, we introduce a heteroskedasticity-aware extension of quantile loss that uses input-dependent label smoothing as a potential remedy and improves over quantile loss on several backbones. Overall, our results provide a modern empirical picture of regression objectives for tabular deep learning and highlight how differences in optimal stopping times across uncertainty regions can limit the effectiveness of global early stopping. The code is available at https://github.com/yandex-research/tabular-regression.