mlsanityData to 14 Oct 2026

Fine-Tuning Vision-Language Models for Neutrino Event Analysis in High-Energy Physics Experiments

arXiv 2508.19376 ↗First posted 26 Aug 2025cs.LG cs.AI cs.CVcited by 0

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Recent progress in large language models (LLMs) has shown strong potential for multimodal reasoning beyond natural language. In this work, we explore the use of a fine-tuned Vision-Language Model (VLM), based on LLaMA 3.2, for classifying neutrino interactions from pixelated detector images in high-energy physics (HEP) experiments. We benchmark its performance against an established CNN baseline used in experiments like NOvA and DUNE, evaluating metrics such as classification accuracy, precision, recall, and AUC-ROC. Our results show that the VLM not only matches or exceeds CNN performance but also enables richer reasoning and better integration of auxiliary textual or semantic context. These findings suggest that VLMs offer a promising general-purpose backbone for event classification in HEP, paving the way for multimodal approaches in experimental neutrino physics.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 17 of its 34 references that are in the map. First posted on arXiv 26 Aug 2025.

Builds on

17 of 34 references.

References · 17

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2017On Calibration of Modern Neural Networkscited by 4,960
  5. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  6. 2017Squeeze-and-Excitation Networkscited by 4,789
  7. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  8. 2019Language Models as Knowledge Bases?cited by 1,936
  9. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  10. 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
  11. 2017Lexically Constrained Decoding for Sequence Generation Using Grid Beam Searchcited by 237
  12. 2023Multimodal Large Language Models: A Surveycited by 195
5 more
  1. 2016Parameterized Machine Learning for High-Energy Physicscited by 34
  2. 2016A Convolutional Neural Network Neutrino Event Classifiercited by 26
  3. 2023Reconstruction of Unstable Heavy Particles Using Deep Symmetry-Preserving Attention Networkscited by 7
  4. 2017Application of a Convolutional Neural Network for image classification to the analysis of collisions in High Energy Physicscited by 4
  5. 2025Particle Hit Clustering and Identification Using Point Set Transformers in Liquid Argon Time Projection Chamberscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Adapting Vision-Language Models for Neutrino Event Classification in High-Energy Physics
  2. 2026Finding and using interpretable latents in a neutrino foundation model with sparse autoencoders
  3. 2025Towards Foundation Models for Experimental Readout Systems Combining Discrete and Continuous Data
  4. 2025Scaling Large Vision-Language Models for Enhanced Multimodal Comprehension In Biomedical Image Analysis
  5. 2025Panda: Self-distillation of Reusable Sensor-level Representations for High Energy Physics
  6. 2024Inducing High Energy-Latency of Large Vision-Language Models with Verbose Images
  7. 2025ExpVid: A Benchmark for Experiment Video Understanding & Reasoning
  8. 2024Discovering the Gems in Early Layers: Accelerating Long-Context LLMs with 1000x Input Token Reduction