# Marginal Effects for Non-Linear Prediction Functions

arXiv: https://arxiv.org/abs/2201.08837 · HTML: https://mlsanity.com/paper/2201.08837/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 67 references that are in the map.
- First posted on arXiv 21 Jan 2022.

## Builds on (most cited first, 12 of 12)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods](https://mlsanity.com/paper/1911.02508/) (2019)
- [Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models](https://mlsanity.com/paper/1612.08468/) (2016)
- [Unrestricted Permutation forces Extrapolation: Variable Importance Requires at least One More Model, or There Is No Free Variable Importance](https://mlsanity.com/paper/1905.03151/) (2019)
- [General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Models](https://mlsanity.com/paper/2007.04131/) (2020)
- [S-LIME: Stabilized-LIME for Model Explanation](https://mlsanity.com/paper/2106.07875/) (2021)
- [Model-based Recursive Partitioning for Subgroup Analyses](https://mlsanity.com/paper/1605.04078/) (2016)
- [Quantifying Model Complexity via Functional Decomposition for Better Post-Hoc Interpretability](https://mlsanity.com/paper/1904.03867/) (2019)
- [Efficient non-greedy optimization of decision trees](https://mlsanity.com/paper/1511.04056/) (2015)
- [Approximation Trees: Statistical Stability in Model Distillation](https://mlsanity.com/paper/1808.07573/) (2018)

## Built on by (most cited first, 5 of 5)

- [cito: An R package for training neural networks using torch](https://mlsanity.com/paper/2303.09599/) (2023)
- [Effector: A Python package for regional explanations](https://mlsanity.com/paper/2404.02629/) (2024)
- [Are Logistic Models Really Interpretable?](https://mlsanity.com/paper/2406.13427/) (2024)
- [fmeffects: An R Package for Forward Marginal Effects](https://mlsanity.com/paper/2310.02008/) (2023)
- [NIMO: a Nonlinear Interpretable MOdel](https://mlsanity.com/paper/2506.05059/) (2025)

## Abstract (arXiv)

Beta coefficients for linear regression models represent the ideal form of an interpretable feature effect. However, for non-linear models and especially generalized linear models, the estimated coefficients cannot be interpreted as a direct feature effect on the predicted outcome. Hence, marginal effects are typically used as approximations for feature effects, either in the shape of derivatives of the prediction function or forward differences in prediction due to a change in a feature value. While marginal effects are commonly used in many scientific fields, they have not yet been adopted as a model-agnostic interpretation method for machine learning models. This may stem from their inflexibility as a univariate feature effect and their inability to deal with the non-linearities found in black box models. We introduce a new class of marginal effects termed forward marginal effects. We argue to abandon derivatives in favor of better-interpretable forward differences. Furthermore, we generalize marginal effects based on forward differences to multivariate changes in feature values. To account for the non-linearity of prediction functions, we introduce a non-linearity measure for marginal effects. We argue against summarizing feature effects of a non-linear prediction function in a single metric such as the average marginal effect. Instead, we propose to partition the feature space to compute conditional average marginal effects on feature subspaces, which serve as conditional feature effect estimates.