# Dissecting Jet-Tagger Through Mechanistic Interpretability

arXiv: https://arxiv.org/abs/2605.09881 · HTML: https://mlsanity.com/paper/2605.09881/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability.
- Builds on 18 of its 48 references that are in the map.
- First posted on arXiv 11 May 2026.

## Placed in

- Mechanistic interpretability / [Circuits and mechanistic analysis](https://mlsanity.com/mechinterp/circuits-and-mechanistic-analysis/)

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [NormFormer: Improved Transformer Pretraining with Extra Normalization](https://mlsanity.com/paper/2110.09456/) (2021)
- [ParticleNet: Jet Tagging via Particle Clouds](https://mlsanity.com/paper/1902.08570/) (2019)
- [Jet-Images -- Deep Learning Edition](https://mlsanity.com/paper/1511.05190/) (2015)
- [Lecture notes on ridge regression](https://mlsanity.com/paper/1509.09169/) (2015)
- [Explainable AI for ML jet taggers using expert variables and layerwise relevance propagation](https://mlsanity.com/paper/2011.13466/) (2020)
- [Explaining machine-learned particle-flow reconstruction](https://mlsanity.com/paper/2111.12840/) (2021)
- [Graph Neural Networks in Particle Physics: Implementations, Innovations, and Challenges](https://mlsanity.com/paper/2203.12852/) (2022)
- [A Detailed Study of Interpretability of Deep Neural Network based Top Taggers](https://mlsanity.com/paper/2210.04371/) (2022)
- [Do graph neural networks learn traditional jet substructure?](https://mlsanity.com/paper/2211.09912/) (2022)
- [Mixture-of-Experts Graph Transformers for Interpretable Particle Collision Detection](https://mlsanity.com/paper/2501.03432/) (2025)
- [Evidential Deep Learning for Uncertainty Quantification and Out-of-Distribution Detection in Jet Identification using Deep Neural Networks](https://mlsanity.com/paper/2501.05656/) (2025)
- [Explainable AI for Jet Tagging: A Comparative Study of GNNExplainer, GNNShap, and GradCAM for Jet Tagging in the Lund Jet Plane](https://mlsanity.com/paper/2604.25885/) (2026)
- [Stable and Interpretable Jet Physics with IRC-Safe Equivariant Feature Extraction](https://mlsanity.com/paper/2509.22059/) (2025)

## Abstract (arXiv)

Mechanistic interpretability seeks to reverse engineer a trained neural network by identifying the minimal subset of internal components. We perform a mechanistic interpretability analysis of the Particle Transformer architecture, trained on the Top Quark Tagging reference dataset, with the goal of identifying the computational circuit responsible for jet classification and characterizing the physical content of its internal representations. Combining zero ablation, path patching with two complementary on-manifold corruption strategies and linear probing of the residual stream, we identify a sparse six-head circuit that recovers the great majority of the full model performance while admitting a clean source-relay-readout interpretation. In this circuit, a single early layer head serves as the primary causal source, a cluster of middle-layer heads acts as relays selectively attending to hard pairwise substructure and a single late-layer head reads out the aggregated signal. Linear probes show that the residual stream is preferentially aligned with the energy correlator basis over the N-subjettiness basis. Within the energy correlator basis, the model preferentially encodes 2-prong substructure observables over the 3-prong observables. A per-layer trained probe further reveals that the apparent single step commitment of the model to a classification decision in the first class attention block is in fact a basis rotation, with the discriminating signal already saturating in the particle attention stack. These results demonstrate that mechanistic interpretability methods developed for natural language models can be used for jet physics classifiers and indicate that gradient descent may rediscover physically meaningful aspects of jet tagging without supervision.