# Interpretability, Then What? Editing Machine Learning Models to Reflect Human Knowledge and Values

arXiv: https://arxiv.org/abs/2206.15465 · HTML: https://mlsanity.com/paper/2206.15465/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 30 references that are in the map.
- First posted on arXiv 30 Jun 2022.

## Builds on (most cited first, 13 of 13)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Neural Additive Models: Interpretable Machine Learning with Neural Nets](https://mlsanity.com/paper/2004.13912/) (2020)
- [Understanding the Role of Individual Units in a Deep Neural Network](https://mlsanity.com/paper/2009.05041/) (2020)
- [InterpretML: A Unified Framework for Machine Learning Interpretability](https://mlsanity.com/paper/1909.09223/) (2019)
- [Identifying and Controlling Important Neurons in Neural Machine Translation](https://mlsanity.com/paper/1811.01157/) (2018)
- [Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs](https://mlsanity.com/paper/2004.11440/) (2020)
- [Interpretable Classification Models for Recidivism Prediction](https://mlsanity.com/paper/1503.07810/) (2015)
- [How Interpretable and Trustworthy are GAMs?](https://mlsanity.com/paper/2006.06466/) (2020)
- [In Pursuit of Interpretable, Fair and Accurate Machine Learning for Criminal Recidivism Prediction](https://mlsanity.com/paper/2005.04176/) (2020)
- [Finding Experts in Transformer Models](https://mlsanity.com/paper/2005.07647/) (2020)
- [GAM Changer: Editing Generalized Additive Models with Interactive Visualization](https://mlsanity.com/paper/2112.03245/) (2021)
- [Scalable visualisation methods for modern Generalized Additive Models](https://mlsanity.com/paper/1809.10632/) (2018)

## Built on by (most cited first, 25 of 25)

- [From Shapley Values to Generalized Additive Models and back](https://mlsanity.com/paper/2209.04012/) (2022)
- [Farsight: Fostering Responsible AI Awareness During AI Application Prototyping](https://mlsanity.com/paper/2402.15350/) (2024)
- [EXMOS: Explanatory Model Steering Through Multifaceted Explanations and Data Configurations](https://mlsanity.com/paper/2402.00491/) (2024)
- [TimberTrek: Exploring and Curating Sparse Decision Trees with Interactive Visualization](https://mlsanity.com/paper/2209.09227/) (2022)
- [Where Are We So Far? Understanding Data Storytelling Tools from the Perspective of Human-AI Collaboration](https://mlsanity.com/paper/2309.15723/) (2023)
- [MeMemo: On-device Retrieval Augmentation for Private and Personalized Text Generation](https://mlsanity.com/paper/2407.01972/) (2024)
- [Parameter-tuning-free data entry error unlearning with adaptive selective synaptic dampening](https://mlsanity.com/paper/2402.10098/) (2024)
- [GAM Coach: Towards Interactive and User-centered Algorithmic Recourse](https://mlsanity.com/paper/2302.14165/) (2023)
- [AI, Meet Human: Learning Paradigms for Hybrid Decision Making Systems](https://mlsanity.com/paper/2402.06287/) (2024)
- [Explainable AI needs formalization](https://mlsanity.com/paper/2409.14590/) (2024)
- [Lessons Learned from EXMOS User Studies: A Technical Report Summarizing Key Takeaways from User Studies Conducted to Evaluate The EXMOS Platform](https://mlsanity.com/paper/2310.02063/) (2023)
- [Fast and Interpretable Mortality Risk Scores for Critical Care Patients](https://mlsanity.com/paper/2311.13015/) (2023)
- [An Explanatory Model Steering System for Collaboration between Domain Experts and AI](https://mlsanity.com/paper/2405.13038/) (2024)
- [SuperNOVA: Design Strategies and Opportunities for Interactive Visualization in Computational Notebooks](https://mlsanity.com/paper/2305.03039/) (2023)
- [Interactive Generalized Additive Model and Its Applications in Electric Load Forecasting](https://mlsanity.com/paper/2310.15662/) (2023)
- [WebSHAP: Towards Explaining Any Machine Learning Models Anywhere](https://mlsanity.com/paper/2303.09545/) (2023)
- [DeforestVis: Behavior Analysis of Machine Learning Models with Surrogate Decision Stumps](https://mlsanity.com/paper/2304.00133/) (2023)
- [Overcoming Algorithm Aversion with Transparency: Can Transparent Predictions Change User Behavior?](https://mlsanity.com/paper/2508.03168/) (2025)
- [Learning to Advise Humans in High-Stakes Settings](https://mlsanity.com/paper/2210.12849/) (2022)
- [Lessons from the Development of an Anomaly Detection Interface on the Mars Perseverance Rover using the ISHMAP Framework](https://mlsanity.com/paper/2302.07187/) (2023)
- [Towards Feminist Intersectional XAI: From Explainability to Response-Ability](https://mlsanity.com/paper/2305.03375/) (2023)
- [The Value of AI Advice: Personalized and Value-Maximizing AI Advisors Are Necessary to Reliably Benefit Experts and Organizations](https://mlsanity.com/paper/2412.19530/) (2024)
- [iTARGET: Interpretable Tailored Age Regression for Grouped Epigenetic Traits](https://mlsanity.com/paper/2501.02401/) (2025)
- [Position: We Need Practical AI Alignment Methods to Mirror Human Reasoning](https://mlsanity.com/paper/2608.12372/) (2026)
- [Target leakage, not model class, explains reported accuracy in survey-based cardiovascular screening: a leakage-tiered audit of glass-box and tabular foundation models](https://mlsanity.com/paper/2609.11838/) (2026)

## Abstract (arXiv)

Machine learning (ML) interpretability techniques can reveal undesirable patterns in data that models exploit to make predictions--potentially causing harms once deployed. However, how to take action to address these patterns is not always clear. In a collaboration between ML and human-computer interaction researchers, physicians, and data scientists, we develop GAM Changer, the first interactive system to help domain experts and data scientists easily and responsibly edit Generalized Additive Models (GAMs) and fix problematic patterns. With novel interaction techniques, our tool puts interpretability into action--empowering users to analyze, validate, and align model behaviors with their knowledge and values. Physicians have started to use our tool to investigate and fix pneumonia and sepsis risk prediction models, and an evaluation with 7 data scientists working in diverse domains highlights that our tool is easy to use, meets their model editing needs, and fits into their current workflows. Built with modern web technologies, our tool runs locally in users' web browsers or computational notebooks, lowering the barrier to use. GAM Changer is available at the following public demo link: https://interpret.ml/gam-changer.