# Interpretable models for extrapolation in scientific machine learning

arXiv: https://arxiv.org/abs/2212.10283 · HTML: https://mlsanity.com/paper/2212.10283/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 61 references that are in the map.
- First posted on arXiv 16 Dec 2022.

## Builds on (most cited first, 7 of 7)

- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Molecular Graph Convolutions: Moving Beyond Fingerprints](https://mlsanity.com/paper/1603.00856/) (2016)
- [Problems with Shapley-value-based explanations as feature importance measures](https://mlsanity.com/paper/2002.11097/) (2020)
- [Challenges in Deploying Machine Learning: a Survey of Case Studies](https://mlsanity.com/paper/2011.09926/) (2020)
- [Interpretable Classification Models for Recidivism Prediction](https://mlsanity.com/paper/1503.07810/) (2015)
- [Machine learning modeling of superconducting critical temperature](https://mlsanity.com/paper/1709.02727/) (2017)
- [High-Dimensional Materials and Process Optimization using Data-driven Experimental Design with Well-Calibrated Uncertainty Estimates](https://mlsanity.com/paper/1704.07423/) (2017)

## Built on by (most cited first, 11 of 11)

- [Multimodal Foundation Models for Material Property Prediction and Discovery](https://mlsanity.com/paper/2312.00111/) (2023)
- [Interpreting and generalizing deep learning in physics-based problems with functional linear models](https://mlsanity.com/paper/2307.04569/) (2023)
- [Known Unknowns: Out-of-Distribution Property Prediction in Materials and Molecules](https://mlsanity.com/paper/2502.05970/) (2025)
- [Large Language Models for Extrapolative Modeling of Manufacturing Processes](https://mlsanity.com/paper/2502.12185/) (2025)
- [Geometry Matters: Benchmarking Scientific ML Approaches for Flow Prediction around Complex Geometries](https://mlsanity.com/paper/2501.01453/) (2024)
- [Predicting band gap from chemical composition: A simple learned model for a material property with atypical statistics](https://mlsanity.com/paper/2501.02932/) (2025)
- [Learning simple heuristic rules for classifying materials based on chemical composition](https://mlsanity.com/paper/2505.02361/) (2025)
- [Building Trustworthy AI for Materials Discovery: From Autonomous Laboratories to Z-scores](https://mlsanity.com/paper/2512.01080/) (2025)
- [Symbolic Learning for Material Discovery](https://mlsanity.com/paper/2312.11487/) (2023)
- [Materials-Discovery Workflows Guided by Symbolic Regression: Identifying Acid-Stable Oxides for Electrocatalysis](https://mlsanity.com/paper/2412.05947/) (2024)
- [Sparse mixed linear modeling with anchor-based guidance for high-entropy alloy discovery](https://mlsanity.com/paper/2504.20354/) (2025)

## Abstract (arXiv)

Data-driven models are central to scientific discovery. In efforts to achieve state-of-the-art model accuracy, researchers are employing increasingly complex machine learning algorithms that often outperform simple regressions in interpolative settings (e.g. random k-fold cross-validation) but suffer from poor extrapolation performance, portability, and human interpretability, which limits their potential for facilitating novel scientific insight. Here we examine the trade-off between model performance and interpretability across a broad range of science and engineering problems with an emphasis on materials science datasets. We compare the performance of black box random forest and neural network machine learning algorithms to that of single-feature linear regressions which are fitted using interpretable input features discovered by a simple random search algorithm. For interpolation problems, the average prediction errors of linear regressions were twice as high as those of black box models. Remarkably, when prediction tasks required extrapolation, linear models yielded average error only 5% higher than that of black box models, and outperformed black box models in roughly 40% of the tested prediction tasks, which suggests that they may be desirable over complex algorithms in many extrapolation problems because of their superior interpretability, computational overhead, and ease of use. The results challenge the common assumption that extrapolative models for scientific machine learning are constrained by an inherent trade-off between performance and interpretability.