# The Need for Interpretable Features: Motivation and Taxonomy

arXiv: https://arxiv.org/abs/2202.11748 · HTML: https://mlsanity.com/paper/2202.11748/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 39 references that are in the map.
- First posted on arXiv 23 Feb 2022.

## Builds on (most cited first, 16 of 16)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Explainable Machine Learning in Deployment](https://mlsanity.com/paper/1909.06342/) (2019)
- [Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challenges](https://mlsanity.com/paper/2010.09337/) (2020)
- [Feature Engineering for Predictive Modeling using Reinforcement Learning](https://mlsanity.com/paper/1709.07150/) (2017)
- [Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs](https://mlsanity.com/paper/2004.11440/) (2020)
- [Stakeholders in Explainable AI](https://mlsanity.com/paper/1810.00184/) (2018)
- [Interpretable and Differentially Private Predictions](https://mlsanity.com/paper/1906.02004/) (2019)
- [Sibyl: Understanding and Addressing the Usability Challenges of Machine Learning In High-Stakes Decision Making](https://mlsanity.com/paper/2103.02071/) (2021)
- [Ten Social Dimensions of Conversations and Relationships](https://mlsanity.com/paper/2001.09954/) (2020)
- [Dissonance Between Human and Machine Understanding](https://mlsanity.com/paper/2101.07337/) (2021)
- [The POLAR Framework: Polar Opposites Enable Interpretability of Pre-Trained Word Embeddings](https://mlsanity.com/paper/2001.09876/) (2020)
- [VBridge: Connecting the Dots Between Features and Data to Explain Healthcare Models](https://mlsanity.com/paper/2108.02550/) (2021)
- [MTV: Visual Analytics for Detecting, Investigating, and Annotating Anomalies in Multivariate Time Series](https://mlsanity.com/paper/2112.05734/) (2021)
- [Towards Feature Engineering at Scale for Data from Massive Open Online Courses](https://mlsanity.com/paper/1407.5238/) (2014)

## Built on by (most cited first, 3 of 3)

- [Diagrammatization and Abduction to Improve AI Interpretability With Domain-Aligned Explanations for Medical Diagnosis](https://mlsanity.com/paper/2302.01241/) (2023)
- [Calibrating Practical Privacy Risks for Differentially Private Machine Learning](https://mlsanity.com/paper/2410.22673/) (2024)
- [Functional Groups are All you Need for Chemically Interpretable Molecular Property Prediction](https://mlsanity.com/paper/2509.09619/) (2025)

## Abstract (arXiv)

Through extensive experience developing and explaining machine learning (ML) applications for real-world domains, we have learned that ML models are only as interpretable as their features. Even simple, highly interpretable model types such as regression models can be difficult or impossible to understand if they use uninterpretable features. Different users, especially those using ML models for decision-making in their domains, may require different levels and types of feature interpretability. Furthermore, based on our experiences, we claim that the term "interpretable feature" is not specific nor detailed enough to capture the full extent to which features impact the usefulness of ML explanations. In this paper, we motivate and discuss three key lessons: 1) more attention should be given to what we refer to as the interpretable feature space, or the state of features that are useful to domain experts taking real-world actions, 2) a formal taxonomy is needed of the feature properties that may be required by these domain experts (we propose a partial taxonomy in this paper), and 3) transforms that take data from the model-ready state to an interpretable form are just as essential as traditional ML transforms that prepare features for the model.