Panda: Self-distillation of Reusable Sensor-level Representations for High Energy Physics
On the map: Knowledge distillation → Detection and dense prediction distillation
Abstract (arXiv)
Liquid argon time projection chambers (LArTPCs) provide dense, high-fidelity 3D measurements of particle interactions and underpin current and future neutrino and rare-event experiments. Physics reconstruction typically relies on complex detector-specific pipelines that use tens of hand-engineered pattern recognition algorithms or cascades of task-specific neural networks that require extensive, labeled simulation that requires a careful, time-consuming calibration process. We introduce Panda, a model that learns reusable sensor-level representations directly from raw unlabeled LArTPC data. Panda couples a hierarchical sparse 3D encoder with a multi-view, prototype-based self-distillation objective. On a simulated dataset, Panda substantially improves label efficiency and reconstruction quality, beating the previous state-of-the-art semantic segmentation model with 1,000× fewer labels. We also show that a single set-prediction head 1/20th the size of the backbone with no physical priors trained on frozen outputs from Panda can result in particle identification that is comparable with state-of-the-art (SOTA) reconstruction tools. Full fine-tuning further improves performance across all tasks.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation. Builds on 30 of its 70 references that are in the map. First posted on arXiv 1 Dec 2025.
Builds on
30 of 70 references.
References · 30
- 2017Attention Is All You Needcited by 60,361
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
- 2020End-to-End Object Detection with Transformerscited by 7,414
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
- 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
- 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
- 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
- 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
18 more
- 2021Going deeper with Image Transformerscited by 764
- 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 653
- 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 543
- 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 459
- 2023Hyena Hierarchy: Towards Larger Convolutional Language Modelscited by 393
- 2014Deeply-Supervised Netscited by 367
- 2025Sonata: Self-Supervised Learning of Reliable Point Representationscited by 110
- 2023OneFormer3D: One Transformer for Unified Point Cloud Segmentationcited by 95
- 2024Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Modelscited by 46
- 2025Concerto: Joint 2D-3D Self-Supervised Learning Emerges Spatial Representationscited by 45
- 2024Is Tokenization Needed for Masked Particle Modelling?cited by 22
- 2019Scalable Deep Convolutional Neural Networks for Sparse, Locally Dense Liquid Argon Time Projection Chamber Datacited by 16
- 2025Systems and Algorithms for Convolutional Multi-Hybrid Language Models at Scalecited by 12
- 2021Scalable, End-to-End, Deep-Learning-Based Data Reconstruction Chain for Particle Imaging Detectorscited by 11
- 2025Particle Trajectory Representation Learning with Masked Point Modelingcited by 8
- 2025FM4NPP: A Scaling Foundation Model for Nuclear and Particle Physicscited by 6
- 2025RINO: Renormalization Group Invariance with No Labelscited by 6
- 2020Scalable, Proposal-free Instance Segmentation Network for 3D Pixel Clustering and Particle Trajectory Reconstruction in Liquid Argon Time Projection Chamberscited by 5
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Panda Diplomacy: Foundation Model Pre-training across Particle Imaging Detectors for High Energy and Nuclear Physics
- 2026Finding and using interpretable latents in a neutrino foundation model with sparse autoencoders
- 2026How Architecture and Training Affect TPC Representations Across Experiments
- 2026Comprehensive reconstruction of collider events with hypergraph representation learning and graph-conditioned diffusion
- 2025Scalable Multi-Task Learning for Particle Collision Event Reconstruction with Heterogeneous Graph Neural Networks
- 2025Towards Foundation Models for Experimental Readout Systems Combining Discrete and Continuous Data
- 2026RINO: Renormalization Group Distillation with No Labels
- 2025Even Faster Simulations with Flow Matching: A Study of Zero Degree Calorimeter Responses