# Quantifying Interpretability and Trust in Machine Learning Systems

arXiv: https://arxiv.org/abs/1901.08558 · HTML: https://mlsanity.com/paper/1901.08558/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jan 2019.

## Built on by (most cited first, 30 of 49)

- [The effectiveness of feature attribution methods and its correlation with automatic evaluation scores](https://mlsanity.com/paper/2105.14944/) (2021)
- ["Will You Find These Shortcuts?" A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classification](https://mlsanity.com/paper/2111.07367/) (2021)
- [How can I choose an explainer? An Application-grounded Evaluation of Post-hoc Explanations](https://mlsanity.com/paper/2101.08758/) (2021)
- [Disentanglement via Latent Quantization](https://mlsanity.com/paper/2305.18378/) (2023)
- [Trust in AI: Progress, Challenges, and Future Directions](https://mlsanity.com/paper/2403.14680/) (2024)
- [On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMs](https://mlsanity.com/paper/2407.19200/) (2024)
- [Single-Turn Debate Does Not Help Humans Answer Hard Reading-Comprehension Questions](https://mlsanity.com/paper/2204.05212/) (2022)
- [Do Transformer Models Show Similar Attention Patterns to Task-Specific Human Gaze?](https://mlsanity.com/paper/2205.10226/) (2022)
- [Evaluating Feature Attribution Methods in the Image Domain](https://mlsanity.com/paper/2202.12270/) (2022)
- [Scrutinizing XAI using linear ground-truth data with suppressor variables](https://mlsanity.com/paper/2111.07473/) (2021)
- [Scale Alone Does not Improve Mechanistic Interpretability in Vision Models](https://mlsanity.com/paper/2307.05471/) (2023)
- [On the Lack of Robust Interpretability of Neural Text Classifiers](https://mlsanity.com/paper/2106.04631/) (2021)
- [Two-Turn Debate Doesn't Help Humans Answer Hard Reading Comprehension Questions](https://mlsanity.com/paper/2210.10860/) (2022)
- [Does More Advice Help? The Effects of Second Opinions in AI-Assisted Decision Making](https://mlsanity.com/paper/2401.07058/) (2024)
- [Are we measuring trust correctly in explainability, interpretability, and transparency research?](https://mlsanity.com/paper/2209.00651/) (2022)
- [Rather a Nurse than a Physician -- Contrastive Explanations under Investigation](https://mlsanity.com/paper/2310.11906/) (2023)
- [Shapelet-Based Counterfactual Explanations for Multivariate Time Series](https://mlsanity.com/paper/2208.10462/) (2022)
- [Characterizing Large Language Models as Rationalizers of Knowledge-intensive Tasks](https://mlsanity.com/paper/2311.05085/) (2023)
- [Quality Metrics for Transparent Machine Learning With and Without Humans In the Loop Are Not Correlated](https://mlsanity.com/paper/2107.02033/) (2021)
- [Uncertainty Propagation in XAI: A Comparison of Analytical and Empirical Estimators](https://mlsanity.com/paper/2504.03736/) (2025)
- [Explainable Deep Neural Network for Multimodal ECG Signals: Intermediate vs Late Fusion](https://mlsanity.com/paper/2508.11666/) (2025)
- [Algorithmic Insurance](https://mlsanity.com/paper/2106.00839/) (2021)
- [Double Trouble: How to not explain a text classifier's decisions using counterfactuals synthesized by masked language models?](https://mlsanity.com/paper/2110.11929/) (2021)
- [Less is More: The Influence of Pruning on the Explainability of CNNs](https://mlsanity.com/paper/2302.08878/) (2023)
- [Evaluating self-attention interpretability through human-grounded experimental protocol](https://mlsanity.com/paper/2303.15190/) (2023)
- [The Roles and Modes of Human Interactions with Automated Machine Learning Systems](https://mlsanity.com/paper/2205.04139/) (2022)
- [Predictability and Comprehensibility in Post-Hoc XAI Methods: A User-Centered Analysis](https://mlsanity.com/paper/2309.11987/) (2023)
- [A Turing Test for Transparency](https://mlsanity.com/paper/2106.11394/) (2021)
- [Interpretation Quality Score for Measuring the Quality of interpretability methods](https://mlsanity.com/paper/2205.12254/) (2022)
- [RecRec: Algorithmic Recourse for Recommender Systems](https://mlsanity.com/paper/2308.14916/) (2023)

## Abstract (arXiv)

Decisions by Machine Learning (ML) models have become ubiquitous. Trusting these decisions requires understanding how algorithms take them. Hence interpretability methods for ML are an active focus of research. A central problem in this context is that both the quality of interpretability methods as well as trust in ML predictions are difficult to measure. Yet evaluations, comparisons and improvements of trust and interpretability require quantifiable measures. Here we propose a quantitative measure for the quality of interpretability methods. Based on that we derive a quantitative measure of trust in ML decisions. Building on previous work we propose to measure intuitive understanding of algorithmic decisions using the information transfer rate at which humans replicate ML model predictions. We provide empirical evidence from crowdsourcing experiments that the proposed metric robustly differentiates interpretability methods. The proposed metric also demonstrates the value of interpretability for ML assisted human decision making: in our experiments providing explanations more than doubled productivity in annotation tasks. However unbiased human judgement is critical for doctors, judges, policy makers and others. Here we derive a trust metric that identifies when human decisions are overly biased towards ML predictions. Our results complement existing qualitative work on trust and interpretability by quantifiable measures that can serve as objectives for further improving methods in this field of research.