# Strong Optimal Classification Trees

arXiv: https://arxiv.org/abs/2103.15965 · HTML: https://mlsanity.com/paper/2103.15965/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 93 references that are in the map.
- First posted on arXiv 29 Mar 2021.

## Builds on (most cited first, 28 of 28)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Algorithmic decision making and the cost of fairness](https://mlsanity.com/paper/1701.08230/) (2017)
- [Fairness in Machine Learning: A Survey](https://mlsanity.com/paper/2010.04053/) (2020)
- [Strong mixed-integer programming formulations for trained neural networks](https://mlsanity.com/paper/1811.08359/) (2018)
- [Optimal Sparse Decision Trees](https://mlsanity.com/paper/1904.12847/) (2019)
- [Generalized and Scalable Optimal Sparse Decision Trees](https://mlsanity.com/paper/2006.08690/) (2020)
- [Fairness Under Unawareness: Assessing Disparity When Protected Class Is Unobserved](https://mlsanity.com/paper/1811.11154/) (2018)
- [Learning Optimal and Fair Decision Trees for Non-Discriminative Decision-Making](https://mlsanity.com/paper/1903.10598/) (2019)
- [On Formalizing Fairness in Prediction with Machine Learning](https://mlsanity.com/paper/1710.03184/) (2017)
- [FAHT: An Adaptive Fairness-aware Decision Tree Classifier](https://mlsanity.com/paper/1907.07237/) (2019)
- [Sparse Regression at Scale: Branch-and-Bound rooted in First-Order Optimization](https://mlsanity.com/paper/2004.06152/) (2020)
- [Sparse High-Dimensional Regression: Exact Scalable Algorithms and Phase Transitions](https://mlsanity.com/paper/1709.10029/) (2017)
- [The Voice of Optimization](https://mlsanity.com/paper/1812.09991/) (2018)
- [Optimization of Tree Ensembles](https://mlsanity.com/paper/1705.10883/) (2017)
- [Fair Adversarial Gradient Tree Boosting](https://mlsanity.com/paper/1911.05369/) (2019)
- [Teaching the Old Dog New Tricks: Supervised Learning with Constraints](https://mlsanity.com/paper/2002.10766/) (2020)
- [Interpretable Optimal Stopping](https://mlsanity.com/paper/1812.07211/) (2018)
- [Regularization vs. Relaxation: A conic optimization perspective of statistical variable selection](https://mlsanity.com/paper/1510.06083/) (2015)
- [Principled Deep Neural Network Training through Linear Programming](https://mlsanity.com/paper/1810.03218/) (2018)
- [Sparsity in Optimal Randomized Classification Trees](https://mlsanity.com/paper/2002.09191/) (2020)
- [Optimal Decision Trees for Nonlinear Metrics](https://mlsanity.com/paper/2009.06921/) (2020)
- [Sparse and Smooth Signal Estimation: Convexification of L0 Formulations](https://mlsanity.com/paper/1811.02655/) (2018)
- [Fair Training of Decision Tree Classifiers](https://mlsanity.com/paper/2101.00909/) (2021)
- [Logic-based Benders Decomposition and Binary Decision Diagram Based Approaches for Stochastic Distributed Operating Room Scheduling](https://mlsanity.com/paper/1907.13265/) (2019)
- [Integer Programming, Constraint Programming, and Hybrid Decomposition Approaches to Discretizable Distance Geometry Problems](https://mlsanity.com/paper/1907.12468/) (2019)
- [An Analysis of Regularized Approaches for Constrained Machine Learning](https://mlsanity.com/paper/2005.10674/) (2020)

## Built on by (most cited first, 28 of 28)

- [Quant-BnB: A Scalable Branch-and-Bound Method for Optimal Decision Trees with Continuous Features](https://mlsanity.com/paper/2206.11844/) (2022)
- [Learning Optimal Fair Classification Trees: Trade-offs Between Interpretability, Fairness, and Accuracy](https://mlsanity.com/paper/2201.09932/) (2022)
- [Learning Optimal Prescriptive Trees from Observational Data](https://mlsanity.com/paper/2108.13628/) (2021)
- [Optimization Over Trained Neural Networks: Taking a Relaxing Walk](https://mlsanity.com/paper/2401.03451/) (2024)
- [Scalable Optimal Multiway-Split Decision Trees with Constraints](https://mlsanity.com/paper/2302.06812/) (2023)
- [Optimal or Greedy Decision Trees? Revisiting their Objectives, Tuning, and Performance](https://mlsanity.com/paper/2409.12788/) (2024)
- [Margin Optimal Classification Trees](https://mlsanity.com/paper/2210.10567/) (2022)
- [SORTeD Rashomon Sets of Sparse Decision Trees: Anytime Enumeration](https://mlsanity.com/paper/2511.03344/) (2025)
- [Rule Generation for Classification: Scalability, Interpretability, and Fairness](https://mlsanity.com/paper/2104.10751/) (2021)
- [Mixed integer linear optimization formulations for learning optimal binary classification trees](https://mlsanity.com/paper/2206.04857/) (2022)
- [ODTlearn: A Package for Learning Optimal Decision Trees for Prediction and Prescription](https://mlsanity.com/paper/2307.15691/) (2023)
- [Learning Optimal Classification Trees Robust to Distribution Shifts](https://mlsanity.com/paper/2310.17772/) (2023)
- [Supervised Feature Compression based on Counterfactual Analysis](https://mlsanity.com/paper/2211.09894/) (2022)
- [Tight Mixed-Integer Optimization Formulations for Prescriptive Trees](https://mlsanity.com/paper/2302.14744/) (2023)
- [Rolling Lookahead Learning for Optimal Classification Trees](https://mlsanity.com/paper/2304.10830/) (2023)
- [Soft regression trees: a model variant and a decomposition training algorithm](https://mlsanity.com/paper/2501.05942/) (2025)
- [Experiments with Optimal Model Trees](https://mlsanity.com/paper/2503.12902/) (2025)
- [Optimal Recourse Summaries via Bi-Objective Decision Tree Learning](https://mlsanity.com/paper/2605.07598/) (2026)
- [Loss-Optimal Classification Trees: A Generalized Framework and the Logistic Case](https://mlsanity.com/paper/2306.00857/) (2023)
- [Applications of 0-1 Neural Networks in Prescription and Prediction](https://mlsanity.com/paper/2402.18851/) (2024)
- [Learning Optimal Signal Temporal Logic Decision Trees for Classification: A Max-Flow MILP Formulation](https://mlsanity.com/paper/2407.21090/) (2024)
- [A Unified Optimization Framework for Multiclass Classification with Structured Hyperplane Arrangements](https://mlsanity.com/paper/2510.05047/) (2025)
- [RS-ORT: A Reduced-Space Branch-and-Bound Algorithm for Optimal Regression Trees](https://mlsanity.com/paper/2510.23901/) (2025)
- [Optimal Pattern Detection Tree for Symbolic Rule-Based Classification](https://mlsanity.com/paper/2605.14374/) (2026)
- [Learning Sparse Decision Trees via Transformer Variational Auto-Encoders](https://mlsanity.com/paper/2609.01430/) (2026)
- [Scaling Optimal Classification Trees via Adaptive Feature and Sample Reduction](https://mlsanity.com/paper/2609.05826/) (2026)
- [An Exact Junction-Tree Extended Formulation for Optimal Classification Trees](https://mlsanity.com/paper/2609.24741/) (2026)
- [A Moving-Horizon Approximate Branch-and-Reduce Method for Deep Classification Trees](https://mlsanity.com/paper/2609.38194/) (2026)

## Abstract (arXiv)

Decision trees are among the most popular machine learning models and are used routinely in applications ranging from revenue management and medicine to bioinformatics. In this paper, we consider the problem of learning optimal binary classification trees with univariate splits. Literature on the topic has burgeoned in recent years, motivated both by the empirical suboptimality of heuristic approaches and the tremendous improvements in mixed-integer optimization (MIO) technology. Yet, existing MIO-based approaches from the literature do not leverage the power of MIO to its full extent: they rely on weak formulations, resulting in slow convergence and large optimality gaps. To fill this gap in the literature, we propose an intuitive flow-based MIO formulation for learning optimal binary classification trees. Our formulation can accommodate side constraints to enable the design of interpretable and fair decision trees. Moreover, we show that our formulation has a stronger linear optimization relaxation than existing methods in the case of binary data. We exploit the decomposable structure of our formulation and max-flow/min-cut duality to derive a Benders' decomposition method to speed-up computation. We propose a tailored procedure for solving each decomposed subproblem that provably generates facets of the feasible set of the MIO as constraints to add to the main problem. We conduct extensive computational experiments on standard benchmark datasets on which we show that our proposed approaches are 29 times faster than state-of-the-art MIO-based techniques and improve out-of-sample performance by up to 8%.