mlsanityData to 14 Oct 2026

RINO: Renormalization Group Distillation with No Labels

arXiv 2610.15919 ↗First posted 11 Oct 2026hep-ex cs.AIcited by 0

On the map: Knowledge distillation → General knowledge distillation methods

Abstract (arXiv)

Supervised machine learning in high-energy physics (HEP) requires training on labeled simulations of particle collisions, which are known to be unfaithful to the underlying physics. This leads to significant inaccuracies and biases in the transfer of trained models to real data, limiting the power of deep learning methods. Self-supervised learning (SSL) is a promising alternative for pretraining directly on real, unlabeled data. However, existing techniques largely operate with inductive biases inspired by domains such as images and natural language. We propose RINO (Renormalization Group DIstillation with NO Labels), which, for the first time, exploits the fundamentally multi-scale renormalization group structure of high-energy jet formation as a physically grounded source of SSL views. We define teacher-student representations across energy scales using different levels of hierarchical clustering, aligned via DINO-style self-distillation. To mirror the background-dominated nature of real collider data, RINO pretrains exclusively on jets from quantum chromodynamics (QCD) processes and is evaluated on out-of-distribution (OOD) transfer to rare signal classes. In our controlled data-efficiency study, RINO matches fully supervised training on JetNet after finetuning on only 1% of the labeled data. With the full labeled dataset, RINO improves OOD classification accuracy on top-quark jets by 5.9% and, without retraining, improves AUC on the unseen Higgs boson decay channel H → 4q by 4.1% over the strongest SSL baseline. Scale-structure diagnostics reveal that DINO-style self-distillation permits our backbone to retain scale-dependent structure. Ablation studies show that this accounts for an 8.6% OOD accuracy gap with respect to alternative SSL paradigms such as SimCLR. Code is available at https://github.com/zichunhao/RINO.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation. Builds on 42 of its 81 references that are in the map. First posted on arXiv 11 Oct 2026.

Builds on

42 of 81 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026jBOT: Semantic Jet Representation Clustering Emerges from Self-Distillation16 Jan

Other references · 41

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  8. 2017Neural Discrete Representation Learningcited by 4,712
  9. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  10. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  11. 2020GLU Variants Improve Transformercited by 1,804
  12. 2019Root Mean Square Layer Normalizationcited by 1,480
29 more
  1. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  2. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
  3. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 903
  4. 2023Vision Transformers Need Registerscited by 871
  5. 2021Going deeper with Image Transformerscited by 764
  6. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 739
  7. 2019Using Self-Supervised Learning Can Improve Model Robustness and Uncertaintycited by 514
  8. 2016Interaction Networks for Learning about Objects, Relations and Physicscited by 509
  9. 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 497
  10. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 207
  11. 2018Spreading vectors for similarity searchcited by 118
  12. 2022Particle Transformer for Jet Taggingcited by 98
  13. 2018Energy Flow Networks: Deep Sets for Particle Jetscited by 84
  14. 2021A Living Review of Machine Learning for Particle Physicscited by 78
  15. 2019A critical analysis of self-supervision, or what we can learn from a single imagecited by 76
  16. 2018Machine Learning of coarse-grained Molecular Dynamics Force Fieldscited by 69
  17. 2021Particle Cloud Generation with Message Passing Generative Adversarial Networkscited by 46
  18. 2024Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Modelscited by 46
  19. 2024OmniJet-α: The first cross-task foundation model for particle physicscited by 43
  20. 2024Lorentz-Equivariant Geometric Algebra Transformers for High-Energy Physicscited by 38
  21. 2024A Lorentz-Equivariant Transformer for All of the LHCcited by 28
  22. 2022Guillotine Regularization: Why removing layers is needed to improve generalization in Self-Supervised Learningcited by 26
  23. 2023Explainable Equivariant Neural Networks for Particle Physics: PELICANcited by 24
  24. 2024Re-Simulation-based Self-Supervised Learning for Pre-Training Foundation Modelscited by 22
  25. 2024Is Tokenization Needed for Masked Particle Modelling?cited by 22
  26. 2023How to prepare your task head for finetuningcited by 19
  27. 2025HEP-JEPA: A foundation model for collider physics using joint embedding predictive architecturecited by 13
  28. 2024Learning Symmetry-Independent Jet Representations via Jet-Based Joint Embedding Predictive Architecturecited by 12
  29. 2025Fine-tuning machine-learned particle-flow reconstruction for new detector geometries in future colliderscited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Similarity Pairing with Energy Mover's Distance for Self-Supervised Pre-Training at the LHC
  2. 2023Efficient and Robust Jet Tagging at the LHC with Knowledge Distillation
  3. 2025Panda: Self-distillation of Reusable Sensor-level Representations for High Energy Physics
  4. 2026Pre-Training for Simulation-Based Science: A Study on Jet Foundation Model Training Objectives
  5. 2026Dissecting Jet-Tagger Through Mechanistic Interpretability
  6. 2026Panda Diplomacy: Foundation Model Pre-training across Particle Imaging Detectors for High Energy and Nuclear Physics
  7. 2024Generative Diffusion Models for Fast Simulations of Particle Collisions at CERN
  8. 2025Jet Image Generation in High Energy Physics Using Diffusion Models