# Rethinking Distribution Shifts: Empirical Analysis and Modeling for Tabular Data

arXiv: https://arxiv.org/abs/2307.05284 · HTML: https://mlsanity.com/paper/2307.05284/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jul 2023.

## Built on by (most cited first, 11 of 11)

- [Why Tabular Foundation Models Should Be a Research Priority](https://mlsanity.com/paper/2405.01147/) (2024)
- [A Survey on Evaluation of Out-of-Distribution Generalization](https://mlsanity.com/paper/2403.01874/) (2024)
- [A hierarchical decomposition for explaining ML performance discrepancies](https://mlsanity.com/paper/2402.14254/) (2024)
- [DRO: A Python Library for Distributionally Robust Optimization in Machine Learning](https://mlsanity.com/paper/2505.23565/) (2025)
- [Topology-Aware Dynamic Reweighting for Distribution Shifts on Graph](https://mlsanity.com/paper/2406.01066/) (2024)
- [Domain Generalization Under Posterior Drift](https://mlsanity.com/paper/2510.04441/) (2025)
- [Stability Evaluation via Distributional Perturbation Analysis](https://mlsanity.com/paper/2405.03198/) (2024)
- [Mixed-feature Logistic Regression Robust to Distribution Shifts](https://mlsanity.com/paper/2503.12012/) (2025)
- [Explaining Concept Shift with Interpretable Feature Attribution](https://mlsanity.com/paper/2505.20634/) (2025)
- [Achieving First-Order Statistical Improvements in Data-Driven Optimization: From No-Free-Lunch to Amplified Decision Perturbation](https://mlsanity.com/paper/2608.04312/) (2026)
- [Statistical Properties of Robust Learning under Distributional Shifts](https://mlsanity.com/paper/2608.13133/) (2026)

## Abstract (arXiv)

Different distribution shifts require different interventions, and algorithms must be grounded in the specific shifts they address. However, methodological development for robust algorithms typically relies on structural assumptions that lack empirical validation. Advocating for an empirically grounded data-driven approach to algorithm development, we build an empirical testbed comprising natural shifts across 8 tabular datasets, 172 distribution pairs over 45 methods and 90,000 method configurations encompassing empirical risk minimization and distributionally robust optimization (DRO) methods. We find Y|X-shifts are most prevalent in our testbed, in stark contrast to the heavy focus on X (covariate)-shifts in the ML literature, and that the performance of robust algorithms is no better than that of vanilla methods. To understand why, we conduct an in-depth empirical analysis of DRO methods and find that underlooked implementation details -- such as the choice of underlying model class (e.g., LightGBM) and hyperparameter selection -- have a bigger impact on performance than the ambiguity set or its radius. We illustrate via case studies how a data-driven, inductive understanding of distribution shifts can provide a new approach to algorithm development.