# Contrasting Cost-Agnostic and Cost-Sensitive Losses under Limited Model Capacity via mathcal H-consistency

arXiv: https://arxiv.org/abs/2502.19522 · HTML: https://mlsanity.com/paper/2502.19522/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 48 references that are in the map.
- First posted on arXiv 26 Feb 2025.

## Builds on (most cited first, 13 of 13)

- [Smart "Predict, then Optimize"](https://mlsanity.com/paper/1710.08005/) (2017)
- [Melding the Data-Decisions Pipeline: Decision-Focused Learning for Combinatorial Optimization](https://mlsanity.com/paper/1809.05504/) (2018)
- [The Real-World-Weight Cross-Entropy Loss Function: Modeling the Costs of Mislabeling](https://mlsanity.com/paper/2001.00570/) (2020)
- [Calibration and Consistency of Adversarial Surrogate Losses](https://mlsanity.com/paper/2104.09658/) (2021)
- [Calibrated Surrogate Losses for Adversarially Robust Classification](https://mlsanity.com/paper/2005.13748/) (2020)
- [A Finer Calibration Analysis for Adversarial Robustness](https://mlsanity.com/paper/2105.01550/) (2021)
- [Realizable H-Consistent and Bayes-Consistent Loss Functions for Learning to Defer](https://mlsanity.com/paper/2407.13732/) (2024)
- [Convex Calibration Dimension for Multiclass Loss Matrices](https://mlsanity.com/paper/1408.2764/) (2014)
- [H-Consistency Guarantees for Regression](https://mlsanity.com/paper/2403.19480/) (2024)
- [The Lov'asz Hinge: A Novel Convex Surrogate for Submodular Losses](https://mlsanity.com/paper/1512.07797/) (2015)
- [Surrogate Regret Bounds for Polyhedral Losses](https://mlsanity.com/paper/2110.14031/) (2021)
- [An Embedding Framework for the Design and Analysis of Consistent Polyhedral Surrogates](https://mlsanity.com/paper/2206.14707/) (2022)
- [Decision-Focused Learning without Differentiable Optimization: Learning Locally Optimized Decision Losses](https://mlsanity.com/paper/2203.16067/) (2022)

## Built on by (most cited first, 1 of 1)

- [Establishing Linear Surrogate Regret Bounds for Convex Smooth Losses via Convolutional Fenchel-Young Losses](https://mlsanity.com/paper/2505.09432/) (2025)

## Abstract (arXiv)

There is a prevalent debate in machine learning about whether practitioners should train models to optimize a task-agnostic objective (e.g., cross entropy) or incorporate the downstream decision task into the optimization objective (e.g., weighted cross entropy). In ideal settings, like those with infinite data and infinite model capacity, the two approaches are statistically equivalent for the downstream decision task. In practice, however, incorporating the decision task into model training has been shown to empirically improve task-specific performance in certain real-world scenarios. The cause of these benefits has not been theoretically studied to date. Focusing on the setting with limited model capacity through the model class mathcal H, we establish a strict performance gap between post-processing a model learned with a cost-agnostic objective (e.g., thresholding a risk score prediction) and models learned by optimizing a cost-sensitive model without a threshold search. In particular, we establish this gap when there is a hypothesis recovering the optimal decision boundary for the discrete task, but it does not align with the optimal cost-agnostic hypothesis, and give a simple example demonstrating the plausibility of this setting. With assumptions that are hard to verify in practice, we demonstrate that this gap generally exists on classification datasets from the UCI repository, particularly with very simple models.