# A Framework for Inherently Interpretable Optimization Models

arXiv: https://arxiv.org/abs/2208.12570 · HTML: https://mlsanity.com/paper/2208.12570/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 88 references that are in the map.
- First posted on arXiv 26 Aug 2022.

## Builds on (most cited first, 20 of 20)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Explaining Explanations: An Overview of Interpretability of Machine Learning](https://mlsanity.com/paper/1806.00069/) (2018)
- [European Union regulations on algorithmic decision-making and a "right to explanation"](https://mlsanity.com/paper/1606.08813/) (2016)
- [From Predictive to Prescriptive Analytics](https://mlsanity.com/paper/1402.5481/) (2014)
- [Supersparse Linear Integer Models for Optimized Medical Scoring Systems](https://mlsanity.com/paper/1502.04269/) (2015)
- [Argumentative XAI: A Survey](https://mlsanity.com/paper/2105.11266/) (2021)
- [Contrastive Explanation: A Structural-Model Approach](https://mlsanity.com/paper/1811.03163/) (2018)
- [Interpretable Optimal Stopping](https://mlsanity.com/paper/1812.07211/) (2018)
- [Plan Explanations as Model Reconciliation -- An Empirical Study](https://mlsanity.com/paper/1802.01013/) (2018)
- [K-Adaptability in Two-Stage Mixed-Integer Robust Optimization](https://mlsanity.com/paper/1706.07097/) (2017)
- [Algorithms and Uncertainty Sets for Data-Driven Robust Shortest Path Problems](https://mlsanity.com/paper/1802.04149/) (2018)
- [Argumentation for Explainable Scheduling (Full Paper with Proofs)](https://mlsanity.com/paper/1811.05437/) (2018)
- [Robust Optimization with Decision-Dependent Information Discovery](https://mlsanity.com/paper/2004.08490/) (2020)
- [Effective and interpretable dispatching rules for dynamic job shops via guided empirical learning](https://mlsanity.com/paper/2109.03323/) (2021)
- [Arguing for Decisions: A Qualitative Model of Decision Making](https://mlsanity.com/paper/1302.3560/) (2013)

## Built on by (most cited first, 8 of 8)

- [OptiChat: Bridging Optimization Models and Practitioners with Large Language Models](https://mlsanity.com/paper/2501.08406/) (2025)
- [Counterfactual Explanations for Linear Optimization](https://mlsanity.com/paper/2405.15431/) (2024)
- [A Framework for Data-Driven Explainability in Mathematical Optimization](https://mlsanity.com/paper/2308.08309/) (2023)
- [Evolutionary Computation and Explainable AI: A Roadmap to Understandable Intelligent Systems](https://mlsanity.com/paper/2406.07811/) (2024)
- [Coherent Local Explanations for Mathematical Optimization](https://mlsanity.com/paper/2502.04840/) (2025)
- [Surrogate Interpretable Graph for Random Decision Forests](https://mlsanity.com/paper/2506.01988/) (2025)
- [From approximation error to optimality gap -- Explaining the performance impact of opportunity cost approximation in integrated demand management and vehicle routing](https://mlsanity.com/paper/2412.13851/) (2024)
- [Decision-Value Attribution in Predict-then-Optimize Systems](https://mlsanity.com/paper/2606.29878/) (2026)

## Abstract (arXiv)

With dramatic improvements in optimization software, the solution of large-scale problems that seemed intractable decades ago are now a routine task. This puts even more real-world applications into the reach of optimizers. At the same time, solving optimization problems often turns out to be one of the smaller difficulties when putting solutions into practice. One major barrier is that the optimization software can be perceived as a black box, which may produce solutions of high quality, but can create completely different solutions when circumstances change leading to low acceptance of optimized solutions. Such issues of interpretability and explainability have seen significant attention in other areas, such as machine learning, but less so in optimization. In this paper we propose an optimization framework that inherently comes with an easily interpretable optimization rule, that explains under which circumstances certain solutions are chosen. Focusing on decision trees to represent interpretable optimization rules, we propose integer programming formulations as well as a heuristic method that ensure applicability of our approach even for large-scale problems. Computational experiments using random and real-world data indicate that the costs of inherent interpretability can be very small.