mlsanityData to 14 Oct 2026

SynopticBench: Evaluating Vision-Language Models on Generating Weather Forecast Discussions of the Future

arXiv 2604.16451 ↗First posted 7 Apr 2026cs.CL cs.CV cs.LGcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Recent advances in visual-language models (VLMs) have led to significant improvements in a plethora of complex multimodal tasks like image captioning, report generation, and visual perception. However, generating text from meteorological data is highly challenging because the atmosphere is a chaotic system that is rapidly changing at various spatial and temporal scales. Given the complexity of atmospheric phenomena, it is critical to verifiably quantify the effectiveness of existing VLMs on weather forecasting data. In this work, we present SynopticBench, a high-quality dataset consisting of 1,367,041 text samples of Area Forecast Discussions created by the National Weather Service over the continental United States paired to images of 500mb geopotential height, 2 meter temperature, and 850mb wind velocity in weather forecasts. We also present Synoptic Phenomena Alignment and Coverage Evaluation (SPACE), a novel evaluation framework that can be used to effectively estimate the quality of text descriptions of synoptic weather phenomena. Extensive experiments on generating forecast discussions using state-of-the-art VLMs show the sensitivity of existing evaluation metrics in this domain and enable further exploration into synoptic weather and climate text generation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 4 of its 20 references that are in the map. First posted on arXiv 7 Apr 2026.

Builds on

4 of 20 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024ClimateIQA: A New Dataset and Benchmark to Advance Vision-Language Models in Meteorology Anomalies Analysis14 Jun

Other references · 3

  1. 2024CLLMate: A Multimodal Benchmark for Weather and Climate Events Forecastingcited by 20
  2. 2025Zephyrus: An Agentic Framework for Weather Sciencecited by 10
  3. 2025MeteorPred: A Meteorological Multimodal Large Model and Dataset for Severe Weather Event Predictioncited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025The promising potential of vision language models for the generation of textual weather forecasts
  2. 2025Exploring Multimodal AI Reasoning for Meteorological Forecasting from Skew-T Diagrams
  3. 2026Optimizing Multi-Agent Weather Captioning via Text Gradient Descent: A Training-Free Approach with Consensus-Aware Gradient Fusion
  4. 2026WeatherSyn: An Instruction Tuning MLLM For Weather Forecasting Report Generation
  5. 2025LangPrecip: Language-Aware Multimodal Precipitation Nowcasting
  6. 2026AFDBench: A Reasoning-First AI Scientist for NationalWeather Service Forecast Discussions
  7. 2025ClimaEmpact: Domain-Aligned Small Language Models and Datasets for Extreme Weather Analytics
  8. 2025RadarQA: Multi-modal Quality Analysis of Weather Radar Forecasts