# Explainable AI for ML jet taggers using expert variables and layerwise relevance propagation

arXiv: https://arxiv.org/abs/2011.13466 · HTML: https://mlsanity.com/paper/2011.13466/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Nov 2020.

## Built on by (most cited first, 14 of 14)

- [A Living Review of Machine Learning for Particle Physics](https://mlsanity.com/paper/2102.02770/) (2021)
- [Machine learning with data assimilation and uncertainty quantification for dynamical systems: a review](https://mlsanity.com/paper/2303.10462/) (2023)
- [Interpretable Machine Learning in Physics: A Review](https://mlsanity.com/paper/2503.23616/) (2025)
- [Explaining machine-learned particle-flow reconstruction](https://mlsanity.com/paper/2111.12840/) (2021)
- [Snowmass 2021 Computational Frontier CompF03 Topical Group Report: Machine Learning](https://mlsanity.com/paper/2209.07559/) (2022)
- [A Detailed Study of Interpretability of Deep Neural Network based Top Taggers](https://mlsanity.com/paper/2210.04371/) (2022)
- [Do graph neural networks learn traditional jet substructure?](https://mlsanity.com/paper/2211.09912/) (2022)
- [FAIR AI Models in High Energy Physics](https://mlsanity.com/paper/2212.05081/) (2022)
- [Mixture-of-Experts Graph Transformers for Interpretable Particle Collision Detection](https://mlsanity.com/paper/2501.03432/) (2025)
- [Feature Selection with Distance Correlation](https://mlsanity.com/paper/2212.00046/) (2022)
- [Evidential Deep Learning for Uncertainty Quantification and Out-of-Distribution Detection in Jet Identification using Deep Neural Networks](https://mlsanity.com/paper/2501.05656/) (2025)
- [Stable and Interpretable Jet Physics with IRC-Safe Equivariant Feature Extraction](https://mlsanity.com/paper/2509.22059/) (2025)
- [Towards an Interpretable Data-driven Trigger System for High-throughput Physics Facilities](https://mlsanity.com/paper/2104.06622/) (2021)
- [Dissecting Jet-Tagger Through Mechanistic Interpretability](https://mlsanity.com/paper/2605.09881/) (2026)

## Abstract (arXiv)

A framework is presented to extract and understand decision-making information from a deep neural network (DNN) classifier of jet substructure tagging techniques. The general method studied is to provide expert variables that augment inputs ("eXpert AUGmented" variables, or XAUG variables), then apply layerwise relevance propagation (LRP) to networks both with and without XAUG variables. The XAUG variables are concatenated with the intermediate layers after network-specific operations (such as convolution or recurrence), and used in the final layers of the network. The results of comparing networks with and without the addition of XAUG variables show that XAUG variables can be used to interpret classifier behavior, increase discrimination ability when combined with low-level features, and in some cases capture the behavior of the classifier completely. The LRP technique can be used to find relevant information the network is using, and when combined with the XAUG variables, can be used to rank features, allowing one to find a reduced set of features that capture part of the network performance. In the studies presented, adding XAUG variables to low-level DNNs increased the efficiency of classifiers by as much as 30-40%. In addition to performance improvements, an approach to quantify numerical uncertainties in the training of these DNNs is presented.