# Explainable AI for Jet Tagging: A Comparative Study of GNNExplainer, GNNShap, and GradCAM for Jet Tagging in the Lund Jet Plane

arXiv: https://arxiv.org/abs/2604.25885 · HTML: https://mlsanity.com/paper/2604.25885/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 54 references that are in the map.
- First posted on arXiv 28 Apr 2026.

## Builds on (most cited first, 15 of 15)

- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [GNNExplainer: Generating Explanations for Graph Neural Networks](https://mlsanity.com/paper/1903.03894/) (2019)
- [Explainability in Graph Neural Networks: A Taxonomic Survey](https://mlsanity.com/paper/2012.15445/) (2020)
- [Evaluating Explainability for Graph Neural Networks](https://mlsanity.com/paper/2208.09339/) (2022)
- [ParticleNet: Jet Tagging via Particle Clouds](https://mlsanity.com/paper/1902.08570/) (2019)
- [Jet-Images -- Deep Learning Edition](https://mlsanity.com/paper/1511.05190/) (2015)
- [Explainable Equivariant Neural Networks for Particle Physics: PELICAN](https://mlsanity.com/paper/2307.16506/) (2023)
- [Jet tagging in the Lund plane with graph networks](https://mlsanity.com/paper/2012.08526/) (2020)
- [Deep learning in color: towards automated quark/gluon jet discrimination](https://mlsanity.com/paper/1612.01551/) (2016)
- [Interpretable Machine Learning in Physics: A Review](https://mlsanity.com/paper/2503.23616/) (2025)
- [Graph Neural Networks in Particle Physics: Implementations, Innovations, and Challenges](https://mlsanity.com/paper/2203.12852/) (2022)
- [GNNShap: Scalable and Accurate GNN Explanation using Shapley Values](https://mlsanity.com/paper/2401.04829/) (2024)
- [Interpretable machine learning in Physics](https://mlsanity.com/paper/2203.08021/) (2022)
- [Stable and Interpretable Jet Physics with IRC-Safe Equivariant Feature Extraction](https://mlsanity.com/paper/2509.22059/) (2025)
- [KIGNet: Physics-Motivated Multi-Graph Representation Learning for Explainable Jet Tagging](https://mlsanity.com/paper/2512.07420/) (2025)

## Built on by (most cited first, 3 of 3)

- [Interpreting "Interpretability" and Explaining "Explainability" in Machine Learning in Physics](https://mlsanity.com/paper/2606.26228/) (2026)
- [Dissecting Jet-Tagger Through Mechanistic Interpretability](https://mlsanity.com/paper/2605.09881/) (2026)
- [Finding and using interpretable latents in a neutrino foundation model with sparse autoencoders](https://mlsanity.com/paper/2608.26090/) (2026)

## Abstract (arXiv)

Graph neural networks such as ParticleNet and transformer based networks on point clouds such as ParticleTransformer achieve state-of-the-art performance on jet tagging benchmarks at the Large Hadron Collider, yet the physical reasoning behind their predictions remains opaque. We present different methods, i.e. perturbation-based (GNNExplainer), Shapley-value-based (GNNShap), and gradient-based (GRADCam); adapted to operate on LundNet's Lund-plane graph representation. Leveraging the fact that each node in the Lund plane corresponds to a physically meaningful parton splitting, we construct Monte Carlo truth explanation masks and introduce a physics-informed evaluation framework that goes beyond standard fidelity metrics. We perform the analysis in three transverse-momentum bins (pT ∈ [500,700], [800,1000], and the inclusive region [500,1000] GeV), revealing how explanation quality and focus shift between non-perturbative and perturbative regimes. We further quantify the correlation between explainer-assigned node importance and classical jet substructure observables -- N-subjettiness ratios τ21 and τ32 and the energy correlation functions -- establishing the degree to which the model has learned known QCD features. We find that overall the weight assigned by explainability methods has a correlation with analytic observables, with expected shift across different phase space regimes, indicating that a trained neural network indeed learns some aspects of jet-substructure moments. Our open-source implementation enables reproducible explainability studies for graph-based jet taggers.