mlsanityData to 7 Oct 2026

Omni-Weather: A Unified Multimodal Model for Weather Radar Understanding and Generation

arXiv 2512.21643 ↗First posted 25 Dec 2025cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Weather modeling requires both accurate prediction and mechanistic interpretation, yet existing methods treat these goals in isolation, separating generation from understanding. To address this gap, we present Omni-Weather, the first multimodal foundation model that unifies weather generation and understanding within a single architecture. Omni-Weather integrates a radar encoder for weather generation tasks, followed by unified processing using a shared self-attention mechanism. Moreover, we construct a Chain-of-Thought dataset for causal reasoning in weather generation, enabling interpretable outputs and improved perceptual quality. Extensive experiments show Omni-Weather achieves state-of-the-art performance in both weather generation and understanding. Our findings further indicate that generative and understanding tasks in the weather domain can mutually enhance each other. Omni-Weather also demonstrates the feasibility and value of unifying weather generation and understanding.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 30 of its 37 references that are in the map. First posted on arXiv 25 Dec 2025.

Builds on

30 of 37 references.

References · 30

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  3. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  4. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  5. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
  6. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
  7. 2023ClimaX: A foundation model for weather and climatecited by 325
  8. 2025Transfer between Modalities with MetaQueriescited by 231
  9. 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 205
  10. 2022Earthformer: Exploring Space-Time Transformers for Earth System Forecastingcited by 146
  11. 2024LMFusion: Adapting Pretrained Language Models for Multimodal Generationcited by 120
  12. 2023PreDiff: Precipitation Nowcasting with Latent Diffusion Modelscited by 96
18 more
  1. 2021ClimateBert: A Pretrained Language Model for Climate-Related Textcited by 89
  2. 2023DiffCast: A Unified Framework via Residual Diffusion for Precipitation Nowcastingcited by 57
  3. 2023A Comparative Study of Image Restoration Networks for General Backbone Network Designcited by 56
  4. 2024CasCast: Skillful High-resolution Precipitation Nowcasting via Cascaded Modellingcited by 52
  5. 2025UniGen: Enhanced Training & Test-Time Strategies for Unified Multimodal Understanding and Generationcited by 32
  6. 2024FengWu-GHR: Learning the Kilometer-scale Medium-range Global Weather Forecastingcited by 30
  7. 2024WeatherQA: Can Multimodal Language Models Reason about Severe Weather?cited by 21
  8. 2024Learning A Low-Level Vision Generalist via Visual Task Promptcited by 18
  9. 2023ClimateBERT-NetZero: Detecting and Assessing Net Zero and Reduction Targetscited by 16
  10. 2025Factuality Matters: When Image Generation and Editing Meet Structured Visualscited by 16
  11. 2025Lumina-OmniLV: A Unified Multimodal Framework for General Low-Level Visioncited by 14
  12. 2024WeatherGFM: Learning A Weather Generalist Foundation Model via In-context Learningcited by 12
  13. 2025UniMedVL: Unifying Medical Multimodal Understanding and Generation through Observation-Knowledge-Analysiscited by 12
  14. 2025RadarQA: Multi-modal Quality Analysis of Weather Radar Forecastscited by 8
  15. 2023ClimateNLP: Analyzing Public Sentiment Towards Climate Change Using Natural Language Processingcited by 5
  16. 2024DiffSR: Learning Radar Reflectivity Synthesis via Diffusion Model from Satellite Observationscited by 4
  17. 2024Latent Knowledge-Guided Video Diffusion for Scientific Phenomena Generation from a Single Initial Framecited by 4
  18. 2025Exploring Scalable Unified Modeling for General Low-Level Visioncited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Exploring Multimodal AI Reasoning for Meteorological Forecasting from Skew-T Diagrams
  2. 2026Weather-R1: Logically Consistent Reinforcement Fine-Tuning for Multimodal Reasoning in Meteorology
  3. 2026SynopticBench: Evaluating Vision-Language Models on Generating Weather Forecast Discussions of the Future
  4. 2025Hierarchical AI-Meteorologist: LLM-Agent System for Multi-Scale and Explainable Weather Forecast Reporting
  5. 2026Cyclone: Diffusion Model for Cycle-Consistent Weather Editing from Unpaired Driving Data
  6. 2026AGCD: Agent-Guided Cross-Modal Decoding for Weather Forecasting
  7. 2025The promising potential of vision language models for the generation of textual weather forecasts
  8. 2025LangPrecip: Language-Aware Multimodal Precipitation Nowcasting