# A Holistic Approach to Interpretability in Financial Lending: Models, Visualizations, and Summary-Explanations

arXiv: https://arxiv.org/abs/2106.02605 · HTML: https://mlsanity.com/paper/2106.02605/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 45 references that are in the map.
- First posted on arXiv 4 Jun 2021.

## Builds on (most cited first, 13 of 13)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods](https://mlsanity.com/paper/1911.02508/) (2019)
- [Boolean Decision Rules via Column Generation](https://mlsanity.com/paper/1805.09901/) (2018)
- [Supersparse Linear Integer Models for Optimized Medical Scoring Systems](https://mlsanity.com/paper/1502.04269/) (2015)
- [RuleMatrix: Visualizing and Understanding Classifiers with Rules](https://mlsanity.com/paper/1807.06228/) (2018)
- [The age of secrecy and unfairness in recidivism prediction](https://mlsanity.com/paper/1811.00731/) (2018)
- [Learning Optimized Risk Scores](https://mlsanity.com/paper/1610.00168/) (2016)
- [Interpretable Credit Application Predictions With Counterfactual Explanations](https://mlsanity.com/paper/1811.05245/) (2018)
- [ViCE: Visual Counterfactual Explanations for Machine Learning Models](https://mlsanity.com/paper/2003.02428/) (2020)
- [An Interpretable Model with Globally Consistent Explanations for Credit Risk](https://mlsanity.com/paper/1811.12615/) (2018)
- [Forecasting remaining useful life: Interpretable deep learning approach via variational Bayesian inferences](https://mlsanity.com/paper/1907.05146/) (2019)

## Built on by (most cited first, 16 of 16)

- [Amazing Things Come From Having Many Good Models](https://mlsanity.com/paper/2407.04846/) (2024)
- [Logic-Based Explainability in Machine Learning](https://mlsanity.com/paper/2211.00541/) (2022)
- [Fast Sparse Classification for Generalized Linear and Additive Models](https://mlsanity.com/paper/2202.11389/) (2022)
- [A Survey of Explainable Artificial Intelligence (XAI) in Financial Time Series Forecasting](https://mlsanity.com/paper/2407.15909/) (2024)
- [Minimum Levels of Interpretability for Artificial Moral Agents](https://mlsanity.com/paper/2307.00660/) (2023)
- [Explaining Adverse Actions in Credit Decisions Using Shapley Decomposition](https://mlsanity.com/paper/2204.12365/) (2022)
- [To what extent should we trust AI models when they extrapolate?](https://mlsanity.com/paper/2201.11260/) (2022)
- [An Interpretable Loan Credit Evaluation Method Based on Rule Representation Learner](https://mlsanity.com/paper/2304.00731/) (2023)
- [Are Logistic Models Really Interpretable?](https://mlsanity.com/paper/2406.13427/) (2024)
- [Leo Breiman, the Rashomon Effect, and the Occam Dilemma](https://mlsanity.com/paper/2507.03884/) (2025)
- [Explainable Deep Belief Network based Auto encoder using novel Extended Garson Algorithm](https://mlsanity.com/paper/2207.08501/) (2022)
- [Rethinking Log Odds: Linear Probability Modelling and Expert Advice in Interpretable Machine Learning](https://mlsanity.com/paper/2211.06360/) (2022)
- [Enhancing the Interpretability of Rule-based Explanations through Information Retrieval](https://mlsanity.com/paper/2507.05976/) (2025)
- [Enhancing Credit Risk Prediction: A Multi-stage Ensemble Pipeline](https://mlsanity.com/paper/2509.22381/) (2025)
- [How Simple Can It Get? From Interpretable Equations to Readable Rules for Financial Decision Making](https://mlsanity.com/paper/2608.09433/) (2026)
- [CredWise: A Controlled Agentic Decision-Intelligence Framework for Explainable and Auditable Credit-Risk Assessment](https://mlsanity.com/paper/2609.37223/) (2026)

## Abstract (arXiv)

Lending decisions are usually made with proprietary models that provide minimally acceptable explanations to users. In a future world without such secrecy, what decision support tools would one want to use for justified lending decisions? This question is timely, since the economy has dramatically shifted due to a pandemic, and a massive number of new loans will be necessary in the short term. We propose a framework for such decisions, including a globally interpretable machine learning model, an interactive visualization of it, and several types of summaries and explanations for any given decision. The machine learning model is a two-layer additive risk model, which resembles a two-layer neural network, but is decomposable into subscales. In this model, each node in the first (hidden) layer represents a meaningful subscale model, and all of the nonlinearities are transparent. Our online visualization tool allows exploration of this model, showing precisely how it came to its conclusion. We provide three types of explanations that are simpler than, but consistent with, the global model: case-based reasoning explanations that use neighboring past cases, a set of features that were the most important for the model's prediction, and summary-explanations that provide a customized sparse explanation for any particular lending decision made by the model. Our framework earned the FICO recognition award for the Explainable Machine Learning Challenge, which was the first public challenge in the domain of explainable machine learning.