# Deciding When to Decide: Testing Operational Suboptimality Under Distributional Shift

arXiv: https://arxiv.org/abs/2608.29465 · HTML: https://mlsanity.com/paper/2608.29465/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 57 references that are in the map.
- First posted on arXiv 29 Aug 2026.

## Builds on (most cited first, 24 of 24)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Smart "Predict, then Optimize"](https://mlsanity.com/paper/1710.08005/) (2017)
- [Revisiting Classifier Two-Sample Tests](https://mlsanity.com/paper/1610.06545/) (2016)
- [Melding the Data-Decisions Pipeline: Decision-Focused Learning for Combinatorial Optimization](https://mlsanity.com/paper/1809.05504/) (2018)
- [Differentiation of Blackbox Combinatorial Solvers](https://mlsanity.com/paper/1912.02175/) (2019)
- [From Predictive to Prescriptive Analytics](https://mlsanity.com/paper/1402.5481/) (2014)
- [A Unified View of Label Shift Estimation](https://mlsanity.com/paper/2003.07554/) (2020)
- [Decision-Focused Learning: Through the Lens of Learning to Rank](https://mlsanity.com/paper/2112.03609/) (2021)
- [Classification accuracy as a proxy for two sample testing](https://mlsanity.com/paper/1602.02210/) (2016)
- [Tracking the risk of a deployed model and detecting harmful distribution shifts](https://mlsanity.com/paper/2110.06177/) (2021)
- [Inverse Optimization with Noisy Data](https://mlsanity.com/paper/1507.03266/) (2015)
- [Data-driven Inverse Optimization with Imperfect Information](https://mlsanity.com/paper/1512.05489/) (2015)
- [Data-Driven Estimation in Equilibrium Using Inverse Optimization](https://mlsanity.com/paper/1308.3397/) (2013)
- [Risk Bounds and Calibration for a Smart Predict-then-Optimize Method](https://mlsanity.com/paper/2108.08887/) (2021)
- [A Nonparametric Approach for Multiple Change Point Analysis of Multivariate Data](https://mlsanity.com/paper/1306.4933/) (2013)
- [Gen-DFL: Decision-Focused Generative Learning for Robust Decision Making](https://mlsanity.com/paper/2502.05468/) (2025)
- [Generalization of Change-Point Detection in Time Series Data Based on Direct Density Ratio Estimation](https://mlsanity.com/paper/2001.06386/) (2020)
- [Inverse optimization for the recovery of constraint parameters](https://mlsanity.com/paper/1811.00726/) (2018)
- [Inverse Optimization of Convex Risk Functions](https://mlsanity.com/paper/1607.07099/) (2016)
- [Keeping Medical AI Healthy and Trustworthy: A Review of Detection and Correction Methods for System Degradation](https://mlsanity.com/paper/2506.17442/) (2025)
- [Feasibility-Aware Decision-Focused Learning for Predicting Parameters in the Constraints](https://mlsanity.com/paper/2510.04951/) (2025)
- [Data-Driven Optimization for Atlanta Police Zone Design](https://mlsanity.com/paper/2104.00535/) (2021)
- [Black-Box Optimization with Implicit Constraints for Public Policy](https://mlsanity.com/paper/2310.18449/) (2023)

## Abstract (arXiv)

Deployed decisions are often optimized once and retained because updates impose operational, regulatory, or switching costs. As operating conditions change, when should such decisions be re-optimized? We study this question for stochastic optimization when the objective's functional form is known but the decision maker's trade-offs are encoded by an unknown preference parameter. Standard distribution-shift tests are poorly aligned with this goal: they can flag detectable yet decision-irrelevant changes without determining whether the incumbent decision has become materially suboptimal. We propose textttRADAR (Regret-based Assessment of Decision Adequacy and Risk), a decision-focused framework that uses inverse optimization to infer latent preferences and tests the deployed decision's optimality gap under the current distribution. By targeting regret, textttRADAR ignores decision-irrelevant shifts while detecting changes that warrant re-optimization. We develop two-sample and sequential changepoint procedures and establish asymptotic guarantees for Type-I error and power. Across synthetic optimization problems, a semi-synthetic capacity allocation task, and police-zone planning, textttRADAR more reliably distinguishes harmful from harmless shifts than decision-agnostic alternatives.