Textual interpretation of transient image classifications from large language models
Not on the map. Why?
Abstract (arXiv)
Modern astronomical surveys deliver immense volumes of transient detections, yet distinguishing real astrophysical signals (for example, explosive events) from bogus imaging artefacts remains a challenge. Convolutional neural networks are effectively used for real versus bogus classification; however, their reliance on opaque latent representations hinders interpretability. Here we show that large language models (LLMs) can approach the performance level of a convolutional neural network on three optical transient survey datasets (Pan-STARRS, MeerLICHT and ATLAS) while simultaneously producing direct, human-readable descriptions for every candidate. Using only 15 examples and concise instructions, Google's LLM, Gemini, achieves a 93% average accuracy across datasets that span a range of resolution and pixel scales. We also show that a second LLM can assess the coherence of the output of the first model, enabling iterative refinement by identifying problematic cases. This framework allows users to define the desired classification behaviour through natural language and examples, bypassing traditional training pipelines. Furthermore, by generating textual descriptions of observed features, LLMs enable users to query classifications as if navigating an annotated catalogue, rather than deciphering abstract latent spaces. As next-generation telescopes and surveys further increase the amount of data available, LLM-based classification could help bridge the gap between automated detection and transparent, human-level understanding.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 23 of its 49 references that are in the map. First posted on arXiv 8 Oct 2025.
Builds on
23 of 49 references.
References · 23
- 2017Attention Is All You Needcited by 60,361
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2017Deep reinforcement learning from human preferencescited by 3,912
- 2020Learning to summarize from human feedbackcited by 2,588
- 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
- 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
- 2023Bias and Fairness in Large Language Models: A Surveycited by 639
11 more
- 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 606
- 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 431
- 2024Does Prompt Formatting Have Any Impact on LLM Performance?cited by 141
- 2024Bias in Large Language Models: Origin, Evaluation, and Mitigationcited by 77
- 2023Unleashing the potential of prompt engineering for large language modelscited by 72
- 2020Alert Classification for the ALeRCE Broker System: The Real-time Stamp Classifiercited by 10
- 2017Effective Image Differencing with ConvNets for Real-time Transient Huntingcited by 4
- 2022What's the Difference? The potential for Convolutional Neural Networks for transient detection without template subtractioncited by 4
- 2023Domain Adaptation via Minimax Entropy for Real/Bogus Classification of Astronomical Alertscited by 3
- 2024Enhancing Interpretability Through Loss-Defined Classification Objective in Structured Latent Spacescited by 2
- 2021MeerCRAB: MeerLICHT Classification of Real and Bogus Transients using Deep Learningcited by 1
Built on by
Citing papers per year: 2025: 1; 2026: 2.
3 papers.
- 2026Spec-o3: A Tool-Augmented Vision-Language Agent for Rare Celestial Object Candidate Vetting via Automated Spectral Inspectioncited by 1
- 2025Pre-training vision models for the classification of alerts from wide-field time-domain surveyscited by 0
- 2026Generalist Vision-Language Models for Fast Radio Burst detection: a zero-shot benchmark against a specialized detectorcited by 0
- 2026Generalist Vision-Language Models for Fast Radio Burst detection: a zero-shot benchmark against a specialized detector8 Jul 2026
- 2026Spec-o3: A Tool-Augmented Vision-Language Agent for Rare Celestial Object Candidate Vetting via Automated Spectral Inspection10 Jan 2026
- 2025Pre-training vision models for the classification of alerts from wide-field time-domain surveys12 Dec 2025