# Constrained Classification and Policy Learning

arXiv: https://arxiv.org/abs/2106.12886 · HTML: https://mlsanity.com/paper/2106.12886/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 64 references that are in the map.
- First posted on arXiv 24 Jun 2021.

## Builds on (most cited first, 9 of 9)

- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [A Reductions Approach to Fair Classification](https://mlsanity.com/paper/1803.02453/) (2018)
- [Empirical Risk Minimization under Fairness Constraints](https://mlsanity.com/paper/1802.08626/) (2018)
- [The Frontiers of Fairness in Machine Learning](https://mlsanity.com/paper/1810.08810/) (2018)
- [Interpretable Classification Models for Recidivism Prediction](https://mlsanity.com/paper/1503.07810/) (2015)
- [Counterfactual Risk Minimization: Learning from Logged Bandit Feedback](https://mlsanity.com/paper/1502.02362/) (2015)
- [More Efficient Policy Learning via Optimal Retargeting](https://mlsanity.com/paper/1906.08611/) (2019)
- [Monotonic classification: an overview on algorithms, performance measures and data sets](https://mlsanity.com/paper/1811.07155/) (2018)
- [Interpretable Dynamic Treatment Regimes](https://mlsanity.com/paper/1606.01472/) (2016)

## Built on by (most cited first, 3 of 3)

- [Policy Learning with Distributional Welfare](https://mlsanity.com/paper/2311.15878/) (2023)
- [Econometrics of Machine Learning Methods in Economic Forecasting](https://mlsanity.com/paper/2308.10993/) (2023)
- [Policy Learning with Observational Data: The Case of Hepatitis C Treatment for HIV/HCV Co-Infected Patients](https://mlsanity.com/paper/2605.16593/) (2026)

## Abstract (arXiv)

Modern machine learning approaches to classification, including AdaBoost, support vector machines, and deep neural networks, utilize surrogate loss techniques to circumvent the computational complexity of minimizing empirical classification risk. These techniques are also useful for causal policy learning problems, since estimation of individualized treatment rules can be cast as a weighted (cost-sensitive) classification problem. Consistency of the surrogate loss approaches studied in Zhang (2004) and Bartlett et al. (2006) relies on the assumption of correct specification, which means that the specified set of classifiers is rich enough to contain a first-best classifier. This assumption is, however, less credible when interpretability or fairness constraints restrict the set of classifiers. Consequently, the applicability of surrogate-loss-based algorithms in such second-best scenarios remains unknown. This paper studies the consistency of surrogate loss procedures under a constrained set of classifiers without assuming correct specification. We show that in settings where the constraint restricts the classifier's prediction set only, hinge losses (i.e., ℓ1-support vector machines) are the only surrogate losses that preserve consistency in second-best scenarios. If the constraint additionally restricts the functional form of the classifier, consistency of a surrogate loss approach is not guaranteed, even with hinge loss. We therefore characterize conditions on the constrained set of classifiers that can guarantee consistency of hinge-risk-minimizing classifiers. Exploiting our theoretical results, we develop robust and computationally attractive hinge-loss-based procedures for a monotone classification problem.