# Interpretability in Symbolic Regression: a benchmark of Explanatory Methods using the Feynman data set

arXiv: https://arxiv.org/abs/2404.05908 · HTML: https://mlsanity.com/paper/2404.05908/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 112 references that are in the map.
- First posted on arXiv 8 Apr 2024.

## Builds on (most cited first, 30 of 34)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Explaining Explanations: An Overview of Interpretability of Machine Learning](https://mlsanity.com/paper/1806.00069/) (2018)
- [Towards Robust Interpretability with Self-Explaining Neural Networks](https://mlsanity.com/paper/1806.07538/) (2018)
- [A Benchmark for Interpretability Methods in Deep Neural Networks](https://mlsanity.com/paper/1806.10758/) (2018)
- [A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI](https://mlsanity.com/paper/1907.07374/) (2019)
- [On the Robustness of Interpretability Methods](https://mlsanity.com/paper/1806.08049/) (2018)
- [On the (In)fidelity and Sensitivity for Explanations](https://mlsanity.com/paper/1901.09392/) (2019)
- [InterpretML: A Unified Framework for Machine Learning Interpretability](https://mlsanity.com/paper/1909.09223/) (2019)
- [Contemporary Symbolic Regression Methods and their Relative Performance](https://mlsanity.com/paper/2107.14351/) (2021)
- [Explainable Machine Learning for Scientific Insights and Discoveries](https://mlsanity.com/paper/1905.08883/) (2019)
- [AI Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity](https://mlsanity.com/paper/2006.10782/) (2020)
- [The Dangers of Post-hoc Interpretability: Unjustified Counterfactual Explanations](https://mlsanity.com/paper/1907.09294/) (2019)
- [Benchmarking Attribution Methods with Relative Feature Importance](https://mlsanity.com/paper/1907.09701/) (2019)
- [Automated Experiments on Ad Privacy Settings: A Tale of Opacity, Choice, and Discrimination](https://mlsanity.com/paper/1408.6491/) (2014)
- [General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Models](https://mlsanity.com/paper/2007.04131/) (2020)
- [Interpretability and Explainability: A Machine Learning Zoo Mini-tour](https://mlsanity.com/paper/2012.01805/) (2020)
- [S-LIME: Stabilized-LIME for Model Explanation](https://mlsanity.com/paper/2106.07875/) (2021)
- [Where are we now? A large benchmark study of recent symbolic regression methods](https://mlsanity.com/paper/1804.09331/) (2018)
- ["The Human Body is a Black Box": Supporting Clinical Decision-Making with Deep Learning](https://mlsanity.com/paper/1911.08089/) (2019)
- [Can I Trust the Explainer? Verifying Post-hoc Explanatory Methods](https://mlsanity.com/paper/1910.02065/) (2019)
- [Regularizing Black-box Models for Improved Interpretability](https://mlsanity.com/paper/1902.06787/) (2019)
- [Shape-constrained Symbolic Regression -- Improving Extrapolation with Prior Knowledge](https://mlsanity.com/paper/2103.15624/) (2021)
- [A Greedy Search Tree Heuristic for Symbolic Regression](https://mlsanity.com/paper/1801.01807/) (2018)

## Built on by (most cited first, 3 of 3)

- [Iterated Agent for Symbolic Regression](https://mlsanity.com/paper/2510.08317/) (2025)
- [SPINEX_ Symbolic Regression: Similarity-based Symbolic Regression with Explainable Neighbors Exploration](https://mlsanity.com/paper/2411.03358/) (2024)
- [From Data to Laws: Neural Discovery of Conservation Laws Without False Positives](https://mlsanity.com/paper/2603.20474/) (2026)

## Abstract (arXiv)

In some situations, the interpretability of the machine learning models plays a role as important as the model accuracy. Interpretability comes from the need to trust the prediction model, verify some of its properties, or even enforce them to improve fairness. Many model-agnostic explanatory methods exists to provide explanations for black-box models. In the regression task, the practitioner can use white-boxes or gray-boxes models to achieve more interpretable results, which is the case of symbolic regression. When using an explanatory method, and since interpretability lacks a rigorous definition, there is a need to evaluate and compare the quality and different explainers. This paper proposes a benchmark scheme to evaluate explanatory methods to explain regression models, mainly symbolic regression models. Experiments were performed using 100 physics equations with different interpretable and non-interpretable regression methods and popular explanation methods, evaluating the performance of the explainers performance with several explanation measures. In addition, we further analyzed four benchmarks from the GP community. The results have shown that Symbolic Regression models can be an interesting alternative to white-box and black-box models that is capable of returning accurate models with appropriate explanations. Regarding the explainers, we observed that Partial Effects and SHAP were the most robust explanation models, with Integrated Gradients being unstable only with tree-based models. This benchmark is publicly available for further experiments.