Finding and using interpretable latents in a neutrino foundation model with sparse autoencoders
On the map: Mechanistic interpretability → Sparse autoencoder training and evaluation
Abstract (arXiv)
We present a first application of sparse-autoencoder-based mechanistic interpretability to particle physics. Studying a neutrino foundation model pretrained on IceCube data and fine-tuned for direction reconstruction, we identify a validated atlas of physical concepts in the model representation, using a strict validation protocol consisting of held-out tests, matched nuisance controls, and replication across independent dictionary trainings. Causal interventions show that the direction head barely draws on this atlas. Motivated by this underused information, we train an uncertainty head on the same event-level representation to predict the model's angular reconstruction error. Unlike the direction head, it depends causally on quality and brightness features from the atlas. At 20% selection efficiency, this interpretable estimator improves the median angular resolution from 20.2^circ to 3.2^circ. These results suggest that mechanistic interpretability can reveal learned latent physics encoded within a model's internal representation and help design downstream tasks that exploit it.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Sparse autoencoder training and evaluation, one of 13 directions in Mechanistic interpretability. Builds on 17 of its 34 references that are in the map. First posted on arXiv 26 Aug 2026.
Builds on
17 of 34 references.
Closest ancestors · 2
Same direction, newest first
- 2025Interpretable and Steerable Concept Bottleneck Sparse Autoencoders11 Dec
- 2024Identifying Functionally Important Features with End-to-End Sparse Dictionary Learning17 May
Other references · 15
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
- 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
- 2016Understanding intermediate layers using linear classifier probescited by 1,353
- 2024Scaling and evaluating sparse autoencoderscited by 553
- 2024BatchTopK Sparse Autoencoderscited by 130
- 2025SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretabilitycited by 124
- 2025Are Sparse Autoencoders Useful? A Case Study in Sparse Probingcited by 115
- 2025Sparse Autoencoders Trained on the Same Data Learn Different Featurescited by 93
- 2021A Comprehensive Taxonomy for Explainable Artificial Intelligence: A Systematic Survey of Surveys on Methods and Conceptscited by 91
- 2024Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Modelscited by 46
- 2024Finetuning Foundation Models for Joint Analysis Optimizationcited by 18
3 more
- 2026Neural Scaling Laws for Boosted Jet Taggingcited by 12
- 2026Explainable AI for Jet Tagging: A Comparative Study of GNNExplainer, GNNShap, and GradCAM for Jet Tagging in the Lund Jet Planecited by 3
- 2026Neural posterior estimation of the neutrino direction in IceCube using transformer-encoded normalizing flows on the spherecited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026PhysSAE: Mechanistic Interpretability of PINNs with Sparse Autoencoders
- 2025Fine-Tuning Vision-Language Models for Neutrino Event Analysis in High-Energy Physics Experiments
- 2025Panda: Self-distillation of Reusable Sensor-level Representations for High Energy Physics
- 2025Equivariant Sparse Autoencoders: Mechanistic Interpretability of Neural Networks on Symmetric Data
- 2026Sparse probes and murky physics: a case study of interpretability challenges in a foundation model for continuum dynamics
- 2024Decoding Dark Matter: Specialized Sparse Autoencoders for Interpreting Rare Concepts in Foundation Models
- 2024From Neurons to Neutrons: A Case Study in Interpretability
- 2025Towards Foundation Models for Experimental Readout Systems Combining Discrete and Continuous Data