# Explainable Neural Networks based on Additive Index Models

arXiv: https://arxiv.org/abs/1806.01933 · HTML: https://mlsanity.com/paper/1806.01933/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2018.

## Built on by (most cited first, 30 of 33)

- [How can I choose an explainer? An Application-grounded Evaluation of Post-hoc Explanations](https://mlsanity.com/paper/2101.08758/) (2021)
- [Defining and Quantifying the Emergence of Sparse Concepts in DNNs](https://mlsanity.com/paper/2111.06206/) (2021)
- [Does a Neural Network Really Encode Symbolic Concepts?](https://mlsanity.com/paper/2302.13080/) (2023)
- [Explainable, Interpretable & Trustworthy AI for Intelligent Digital Twin: Case Study on Remaining Useful Life](https://mlsanity.com/paper/2301.06676/) (2023)
- [LocalGLMnet: interpretable deep learning for tabular data](https://mlsanity.com/paper/2107.11059/) (2021)
- [Model Explainability in Deep Learning Based Natural Language Processing](https://mlsanity.com/paper/2106.07410/) (2021)
- [Towards the Dynamics of a DNN Learning Symbolic Interactions](https://mlsanity.com/paper/2407.19198/) (2024)
- [A Survey of Neural Trees](https://mlsanity.com/paper/2209.03415/) (2022)
- [GAM(e) changer or not? An evaluation of interpretable machine learning models based on additive model constraints](https://mlsanity.com/paper/2204.09123/) (2022)
- [Interpretable Machine Learning based on Functional ANOVA Framework: Algorithms and Comparisons](https://mlsanity.com/paper/2305.15670/) (2023)
- [Bias, Fairness, and Accountability with AI and ML Algorithms](https://mlsanity.com/paper/2105.06558/) (2021)
- [Using Model-Based Trees with Boosting to Fit Low-Order Functional ANOVA Models](https://mlsanity.com/paper/2207.06950/) (2022)
- [On marginal feature attributions of tree-based models](https://mlsanity.com/paper/2302.08434/) (2023)
- [FAME: Introducing Fuzzy Additive Models for Explainable AI](https://mlsanity.com/paper/2504.07011/) (2025)
- [Approximation of group explainers with coalition structure using Monte Carlo sampling on the product space of coalitions and features](https://mlsanity.com/paper/2303.10216/) (2023)
- [An AI Architecture with the Capability to Explain Recognition Results](https://mlsanity.com/paper/2406.08740/) (2024)
- [Improving Network Interpretability via Explanation Consistency Evaluation](https://mlsanity.com/paper/2408.04600/) (2024)
- [Less Discriminatory Alternative and Interpretable XGBoost Framework for Binary Classification](https://mlsanity.com/paper/2410.19067/) (2024)
- [Towards interpreting ML-based automated malware detection models: a survey](https://mlsanity.com/paper/2101.06232/) (2021)
- [Monotone Tree-Based GAMI Models by Adapting XGBoost](https://mlsanity.com/paper/2309.02426/) (2023)
- [Are Logistic Models Really Interpretable?](https://mlsanity.com/paper/2406.13427/) (2024)
- [An AI Architecture with the Capability to Classify and Explain Hardware Trojans](https://mlsanity.com/paper/2407.04551/) (2024)
- [Neural Networks and Denotation](https://mlsanity.com/paper/2103.08315/) (2021)
- [Linear Iterative Feature Embedding: An Ensemble Framework for Interpretable Model](https://mlsanity.com/paper/2103.09983/) (2021)
- [Semantic interpretation for convolutional neural networks: What makes a cat a cat?](https://mlsanity.com/paper/2204.07724/) (2022)
- [Interpretable Feature Engineering for Time Series Predictors using Attention Networks](https://mlsanity.com/paper/2205.12723/) (2022)
- [Deep Explainable Learning with Graph Based Data Assessing and Rule Reasoning](https://mlsanity.com/paper/2211.04693/) (2022)
- [Rethinking Log Odds: Linear Probability Modelling and Expert Advice in Interpretable Machine Learning](https://mlsanity.com/paper/2211.06360/) (2022)
- [On the explainability of quantum neural networks based on variational quantum circuits](https://mlsanity.com/paper/2301.05549/) (2023)
- [Sparse Deep Additive Model with Interactions: Enhancing Interpretability and Predictability](https://mlsanity.com/paper/2509.23068/) (2025)

## Abstract (arXiv)

Machine Learning algorithms are increasingly being used in recent years due to their flexibility in model fitting and increased predictive performance. However, the complexity of the models makes them hard for the data analyst to interpret the results and explain them without additional tools. This has led to much research in developing various approaches to understand the model behavior. In this paper, we present the Explainable Neural Network (xNN), a structured neural network designed especially to learn interpretable features. Unlike fully connected neural networks, the features engineered by the xNN can be extracted from the network in a relatively straightforward manner and the results displayed. With appropriate regularization, the xNN provides a parsimonious explanation of the relationship between the features and the output. We illustrate this interpretable feature--engineering property on simulated examples.