# Generalist Vision-Language Models for Fast Radio Burst detection: a zero-shot benchmark against a specialized detector

arXiv: https://arxiv.org/abs/2607.07382 · HTML: https://mlsanity.com/paper/2607.07382/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 70 references that are in the map.
- First posted on arXiv 8 Jul 2026.

## Builds on (most cited first, 15 of 15)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [A Survey on Multimodal Large Language Models](https://mlsanity.com/paper/2306.13549/) (2023)
- [Ablation Studies in Artificial Neural Networks](https://mlsanity.com/paper/1901.08644/) (2019)
- [AstroLLaVA: towards the unification of astronomical data and natural language](https://mlsanity.com/paper/2504.08583/) (2025)
- [AstroMMBench: A Benchmark for Evaluating Multimodal Large Language Models Capabilities in Astronomy](https://mlsanity.com/paper/2510.00063/) (2025)
- [Textual interpretation of transient image classifications from large language models](https://mlsanity.com/paper/2510.06931/) (2025)
- [Radio Astronomy in the Era of Vision-Language Models: Prompt Sensitivity and Adaptation](https://mlsanity.com/paper/2509.02615/) (2025)
- [AstroAlertBench: Evaluating the Accuracy, Reasoning, and Honesty of Multimodal LLMs in Astronomical Classification](https://mlsanity.com/paper/2605.05573/) (2026)
- [A systematic evaluation of vision-language models for observational astronomical reasoning tasks](https://mlsanity.com/paper/2604.24589/) (2026)

## Abstract (arXiv)

Fast Radio Burst (FRB) detection increasingly relies on specialized deep learning models that require large task-specific training sets and cannot be redefined without retraining. We evaluate whether small, open-weight, locally run generalist Vision-Language Models (VLMs) can detect FRBs in dynamic spectra under a zero-shot, prompt-only regime. On a balanced binary benchmark of 2000 simulated L-band spectra, Gemma 4 E2B reaches an accuracy of 94.05%, statistically indistinguishable from the specialized detector SwinYNet (92.85%), with a far lower false-positive rate on structured RFI (4.8% vs. 24.6%) and none on pure noise, though SwinYNet ranks perfectly (ROC-AUC 1.0000 vs. 0.9520). Rewriting the prompt alone reconfigures the same models for three-class FRB/RFI/noise classification, reaching up to 86.0% accuracy without a single false FRB while classifying each 2 s spectrum in 1.0--1.5 s, faster than the observation itself. Applied unchanged to the 1600 real FAST observations of FAST-FREX, they reject real interference almost perfectly (2 and 5 false positives in 1000 negatives) but recover only 28.5% and 27.0% of the 600 catalogued bursts, against 95.7% reported for SwinYNet on the same files. Stratifying those bursts by the dispersed signal in the image shows the limit to be the input representation rather than the classifier, recall rising to 84--85% where the sweep is unambiguous and collapsing to 1% on the 13% of positives carrying no detectable signal in a 2 s undedispersed full-band view. The simulated bursts are nearly 30 times brighter in median, and at matched brightness the recalls agree to within a few points.