# Efficient and Robust Jet Tagging at the LHC with Knowledge Distillation

arXiv: https://arxiv.org/abs/2311.14160 · HTML: https://mlsanity.com/paper/2311.14160/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation.
- Builds on 13 of its 25 references that are in the map.
- First posted on arXiv 23 Nov 2023.

## Placed in

- Knowledge distillation / [General knowledge distillation methods](https://mlsanity.com/distill/general-knowledge-distillation-methods/)

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [TinyBERT: Distilling BERT for Natural Language Understanding](https://mlsanity.com/paper/1909.10351/) (2019)
- [Empirical Evaluation of Rectified Activations in Convolutional Network](https://mlsanity.com/paper/1505.00853/) (2015)
- [Lorentz Group Equivariant Neural Network for Particle Physics](https://mlsanity.com/paper/2006.04780/) (2020)
- [Particle Transformer for Jet Tagging](https://mlsanity.com/paper/2202.03772/) (2022)
- [Energy Flow Networks: Deep Sets for Particle Jets](https://mlsanity.com/paper/1810.05165/) (2018)
- [Transferring Inductive Biases through Knowledge Distillation](https://mlsanity.com/paper/2006.00555/) (2020)
- [Machine Learning for Particle Flow Reconstruction at CMS](https://mlsanity.com/paper/2203.00330/) (2022)
- [Interpretable Joint Event-Particle Reconstruction for Neutrino Physics at NOvA with Sparse CNNs and Transformers](https://mlsanity.com/paper/2303.06201/) (2023)

## Built on by (most cited first, 3 of 3)

- [Real-time Anomaly Detection at the L1 Trigger of CMS Experiment](https://mlsanity.com/paper/2411.19506/) (2024)
- [The Pareto Frontier of Resilient Jet Tagging](https://mlsanity.com/paper/2509.19431/) (2025)
- [Interpreting "Interpretability" and Explaining "Explainability" in Machine Learning in Physics](https://mlsanity.com/paper/2606.26228/) (2026)

## Abstract (arXiv)

The challenging environment of real-time data processing systems at the Large Hadron Collider (LHC) strictly limits the computational complexity of algorithms that can be deployed. For deep learning models, this implies that only models with low computational complexity that have weak inductive bias are feasible. To address this issue, we utilize knowledge distillation to leverage both the performance of large models and the reduced computational complexity of small ones. In this paper, we present an implementation of knowledge distillation, demonstrating an overall boost in the student models' performance for the task of classifying jets at the LHC. Furthermore, by using a teacher model with a strong inductive bias of Lorentz symmetry, we show that we can induce the same inductive bias in the student model which leads to better robustness against arbitrary Lorentz boost.