# InterpreTabNet: Distilling Predictive Signals from Tabular Data by Salient Feature Interpretation

arXiv: https://arxiv.org/abs/2406.00426 · HTML: https://mlsanity.com/paper/2406.00426/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jun 2024.

## Built on by (most cited first, 11 of 11)

- [A Comprehensive Survey on Self-Interpretable Neural Networks](https://mlsanity.com/paper/2501.15638/) (2025)
- [TabRep: Training Tabular Diffusion Models with a Simple and Effective Continuous Representation](https://mlsanity.com/paper/2504.04798/) (2025)
- [Variational Uncertainty Decomposition for In-Context Learning](https://mlsanity.com/paper/2509.02327/) (2025)
- [Birds look like cars: Adversarial analysis of intrinsically interpretable deep learning](https://mlsanity.com/paper/2503.08636/) (2025)
- [Selecting Feature Interactions for Generalized Additive Models by Distilling Foundation Models](https://mlsanity.com/paper/2604.13332/) (2026)
- [Towards Interpretable Deep Neural Networks for Tabular Data](https://mlsanity.com/paper/2509.08617/) (2025)
- [TabRAG: Improving Tabular Document Question Answering for Retrieval Augmented Generation via Structured Representations](https://mlsanity.com/paper/2511.06582/) (2025)
- [ReSS: Learning Reasoning Models for Tabular Data Prediction via Symbolic Scaffold](https://mlsanity.com/paper/2604.13392/) (2026)
- [From Residuals to Reasons: LLM-Guided Mechanism Inference from Tabular Data](https://mlsanity.com/paper/2605.22897/) (2026)
- [Guided Data Generation for Understanding Model Behavior](https://mlsanity.com/paper/2502.06658/) (2025)
- [XNNTab -- Interpretable Neural Networks for Tabular Data using Sparse Autoencoders](https://mlsanity.com/paper/2512.13442/) (2025)

## Abstract (arXiv)

Tabular data are omnipresent in various sectors of industries. Neural networks for tabular data such as TabNet have been proposed to make predictions while leveraging the attention mechanism for interpretability. However, the inferred attention masks are often dense, making it challenging to come up with rationales about the predictive signal. To remedy this, we propose InterpreTabNet, a variant of the TabNet model that models the attention mechanism as a latent variable sampled from a Gumbel-Softmax distribution. This enables us to regularize the model to learn distinct concepts in the attention masks via a KL Divergence regularizer. It prevents overlapping feature selection by promoting sparsity which maximizes the model's efficacy and improves interpretability to determine the important features when predicting the outcome. To assist in the interpretation of feature interdependencies from our model, we employ a large language model (GPT-4) and use prompt engineering to map from the learned feature mask onto natural language text describing the learned signal. Through comprehensive experiments on real-world datasets, we demonstrate that InterpreTabNet outperforms previous methods for interpreting tabular data while attaining competitive accuracy.