# Omni-Weather: A Unified Multimodal Model for Weather Radar Understanding and Generation

arXiv: https://arxiv.org/abs/2512.21643 · HTML: https://mlsanity.com/paper/2512.21643/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 37 references that are in the map.
- First posted on arXiv 25 Dec 2025.

## Builds on (most cited first, 30 of 30)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Emerging Properties in Unified Multimodal Pretraining](https://mlsanity.com/paper/2505.14683/) (2025)
- [Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model](https://mlsanity.com/paper/2408.11039/) (2024)
- [BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Dataset](https://mlsanity.com/paper/2505.09568/) (2025)
- [ClimaX: A foundation model for weather and climate](https://mlsanity.com/paper/2301.10343/) (2023)
- [Transfer between Modalities with MetaQueries](https://mlsanity.com/paper/2504.06256/) (2025)
- [MetaMorph: Multimodal Understanding and Generation via Instruction Tuning](https://mlsanity.com/paper/2412.14164/) (2024)
- [Earthformer: Exploring Space-Time Transformers for Earth System Forecasting](https://mlsanity.com/paper/2207.05833/) (2022)
- [LMFusion: Adapting Pretrained Language Models for Multimodal Generation](https://mlsanity.com/paper/2412.15188/) (2024)
- [PreDiff: Precipitation Nowcasting with Latent Diffusion Models](https://mlsanity.com/paper/2307.10422/) (2023)
- [ClimateBert: A Pretrained Language Model for Climate-Related Text](https://mlsanity.com/paper/2110.12010/) (2021)
- [DiffCast: A Unified Framework via Residual Diffusion for Precipitation Nowcasting](https://mlsanity.com/paper/2312.06734/) (2023)
- [A Comparative Study of Image Restoration Networks for General Backbone Network Design](https://mlsanity.com/paper/2310.11881/) (2023)
- [CasCast: Skillful High-resolution Precipitation Nowcasting via Cascaded Modelling](https://mlsanity.com/paper/2402.04290/) (2024)
- [UniGen: Enhanced Training & Test-Time Strategies for Unified Multimodal Understanding and Generation](https://mlsanity.com/paper/2505.14682/) (2025)
- [FengWu-GHR: Learning the Kilometer-scale Medium-range Global Weather Forecasting](https://mlsanity.com/paper/2402.00059/) (2024)
- [WeatherQA: Can Multimodal Language Models Reason about Severe Weather?](https://mlsanity.com/paper/2406.11217/) (2024)
- [Factuality Matters: When Image Generation and Editing Meet Structured Visuals](https://mlsanity.com/paper/2510.05091/) (2025)
- [Learning A Low-Level Vision Generalist via Visual Task Prompt](https://mlsanity.com/paper/2408.08601/) (2024)
- [ClimateBERT-NetZero: Detecting and Assessing Net Zero and Reduction Targets](https://mlsanity.com/paper/2310.08096/) (2023)
- [Lumina-OmniLV: A Unified Multimodal Framework for General Low-Level Vision](https://mlsanity.com/paper/2504.04903/) (2025)
- [WeatherGFM: Learning A Weather Generalist Foundation Model via In-context Learning](https://mlsanity.com/paper/2411.05420/) (2024)
- [UniMedVL: Unifying Medical Multimodal Understanding and Generation through Observation-Knowledge-Analysis](https://mlsanity.com/paper/2510.15710/) (2025)
- [RadarQA: Multi-modal Quality Analysis of Weather Radar Forecasts](https://mlsanity.com/paper/2508.12291/) (2025)
- [ClimateNLP: Analyzing Public Sentiment Towards Climate Change Using Natural Language Processing](https://mlsanity.com/paper/2310.08099/) (2023)
- [DiffSR: Learning Radar Reflectivity Synthesis via Diffusion Model from Satellite Observations](https://mlsanity.com/paper/2411.06714/) (2024)
- [Latent Knowledge-Guided Video Diffusion for Scientific Phenomena Generation from a Single Initial Frame](https://mlsanity.com/paper/2411.11343/) (2024)
- [Exploring Scalable Unified Modeling for General Low-Level Vision](https://mlsanity.com/paper/2507.14801/) (2025)

## Built on by (most cited first, 2 of 2)

- [SCI-PRM: A Tool Aware Process Reward Model for Scientific Reasoning Verification](https://mlsanity.com/paper/2606.04579/) (2026)
- [S1-Omni: A Unified Multimodal Reasoning Model for Scientific Understanding, Prediction, and Generation](https://mlsanity.com/paper/2607.15686/) (2026)

## Abstract (arXiv)

Weather modeling requires both accurate prediction and mechanistic interpretation, yet existing methods treat these goals in isolation, separating generation from understanding. To address this gap, we present Omni-Weather, the first multimodal foundation model that unifies weather generation and understanding within a single architecture. Omni-Weather integrates a radar encoder for weather generation tasks, followed by unified processing using a shared self-attention mechanism. Moreover, we construct a Chain-of-Thought dataset for causal reasoning in weather generation, enabling interpretable outputs and improved perceptual quality. Extensive experiments show Omni-Weather achieves state-of-the-art performance in both weather generation and understanding. Our findings further indicate that generative and understanding tasks in the weather domain can mutually enhance each other. Omni-Weather also demonstrates the feasibility and value of unifying weather generation and understanding.