# LLMs Understand Glass-Box Models, Discover Surprises, and Suggest Repairs

arXiv: https://arxiv.org/abs/2308.01157 · HTML: https://mlsanity.com/paper/2308.01157/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 27 references that are in the map.
- First posted on arXiv 2 Aug 2023.

## Builds on (most cited first, 14 of 14)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Capabilities of GPT-4 on Medical Challenge Problems](https://mlsanity.com/paper/2303.13375/) (2023)
- [Confounding variables can degrade generalization performance of radiological deep learning models](https://mlsanity.com/paper/1807.00431/) (2018)
- [Language Models are Realistic Tabular Data Generators](https://mlsanity.com/paper/2210.06280/) (2022)
- [TabLLM: Few-shot Classification of Tabular Data with Large Language Models](https://mlsanity.com/paper/2210.10723/) (2022)
- [InterpretML: A Unified Framework for Machine Learning Interpretability](https://mlsanity.com/paper/1909.09223/) (2019)
- [Can Foundation Models Wrangle Your Data?](https://mlsanity.com/paper/2205.09911/) (2022)
- [From Shapley Values to Generalized Additive Models and back](https://mlsanity.com/paper/2209.04012/) (2022)
- [Post-Hoc Explanations Fail to Achieve their Purpose in Adversarial Contexts](https://mlsanity.com/paper/2201.10295/) (2022)
- [GAM Changer: Editing Generalized Additive Models with Interactive Visualization](https://mlsanity.com/paper/2112.03245/) (2021)
- [TabRet: Pre-training Transformer-based Tabular Models for Unseen Columns](https://mlsanity.com/paper/2303.15747/) (2023)
- [Interpretable Medical Diagnostics with Structured Data Extraction by Large Language Models](https://mlsanity.com/paper/2306.05052/) (2023)

## Built on by (most cited first, 7 of 7)

- [Rethinking Interpretability in the Era of Large Language Models](https://mlsanity.com/paper/2402.01761/) (2024)
- [Data Science with LLMs and Interpretable Models](https://mlsanity.com/paper/2402.14474/) (2024)
- [Anomaly Detection and Generation with Diffusion Models: A Survey](https://mlsanity.com/paper/2506.09368/) (2025)
- [Data Analysis in the Era of Generative AI](https://mlsanity.com/paper/2409.18475/) (2024)
- [Large Language Models as Co-Pilots for Causal Inference in Medical Studies](https://mlsanity.com/paper/2407.19118/) (2024)
- [The Architecture of Trust: A Framework for AI-Augmented Real Estate Valuation in the Era of Structured Data](https://mlsanity.com/paper/2508.02765/) (2025)
- [AnTenA: Actionable and Explainable Tensor Analysis System with Large Language Models](https://mlsanity.com/paper/2606.28708/) (2026)

## Abstract (arXiv)

We show that large language models (LLMs) are remarkably good at working with interpretable models that decompose complex outcomes into univariate graph-represented components. By adopting a hierarchical approach to reasoning, LLMs can provide comprehensive model-level summaries without ever requiring the entire model to fit in context. This approach enables LLMs to apply their extensive background knowledge to automate common tasks in data science such as detecting anomalies that contradict prior knowledge, describing potential reasons for the anomalies, and suggesting repairs that would remove the anomalies. We use multiple examples in healthcare to demonstrate the utility of these new capabilities of LLMs, with particular emphasis on Generalized Additive Models (GAMs). Finally, we present the package textttTalkToEBM as an open-source LLM-GAM interface.