# Foundation Models for Weather and Climate Data Understanding: A Comprehensive Survey

arXiv: https://arxiv.org/abs/2312.03014 · HTML: https://mlsanity.com/paper/2312.03014/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 164 of its 345 references that are in the map.
- First posted on arXiv 5 Dec 2023.

## Builds on (most cited first, 30 of 164)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)

## Built on by (most cited first, 30 of 34)

- [Evaluating Large Language Models on Time Series Feature Understanding: A Comprehensive Taxonomy and Benchmark](https://mlsanity.com/paper/2404.16563/) (2024)
- [On the Foundations of Earth and Climate Foundation Models](https://mlsanity.com/paper/2405.04285/) (2024)
- [Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Models](https://mlsanity.com/paper/2402.01749/) (2024)
- [WeatherQA: Can Multimodal Language Models Reason about Severe Weather?](https://mlsanity.com/paper/2406.11217/) (2024)
- [Deep Learning and Foundation Models for Weather Prediction: A Survey](https://mlsanity.com/paper/2501.06907/) (2025)
- [WildfireGPT: Tailored Large Language Model for Wildfire Analysis](https://mlsanity.com/paper/2402.07877/) (2024)
- [Personalized Adapter for Large Meteorology Model on Devices: Towards Weather Foundation Models](https://mlsanity.com/paper/2405.20348/) (2024)
- [Federated Foundation Models on Heterogeneous Time Series](https://mlsanity.com/paper/2412.08906/) (2024)
- [Foundation Models in Remote Sensing: Evolving from Unimodality to Multimodality](https://mlsanity.com/paper/2603.00988/) (2026)
- [Satellite Observations Guided Diffusion Model for Accurate Meteorological States at Arbitrary Resolution](https://mlsanity.com/paper/2502.07814/) (2025)
- [Evaluating and Benchmarking Foundation Models for Earth Observation and Geospatial AI](https://mlsanity.com/paper/2406.18295/) (2024)
- [State-observation augmented diffusion model for nonlinear assimilation with unknown dynamics](https://mlsanity.com/paper/2407.21314/) (2024)
- [Temporal Neural Operator for Modeling Time-Dependent Physical Phenomena](https://mlsanity.com/paper/2504.20249/) (2025)
- [Observation-Guided Meteorological Field Downscaling at Station Scale: A Benchmark and a New Method](https://mlsanity.com/paper/2401.11960/) (2024)
- [SciVid: Cross-Domain Evaluation of Video Models in Scientific Applications](https://mlsanity.com/paper/2507.03578/) (2025)
- [A RAG-Based Multi-Agent LLM System for Natural Hazard Resilience and Adaptation](https://mlsanity.com/paper/2504.17200/) (2025)
- [Interpretable Machine Learning for Weather and Climate Prediction: A Survey](https://mlsanity.com/paper/2403.18864/) (2024)
- [Federated Distillation for Medical Image Classification: Towards Trustworthy Computer-Aided Diagnosis](https://mlsanity.com/paper/2407.02261/) (2024)
- [Decoding complexity: how machine learning is redefining scientific discovery](https://mlsanity.com/paper/2405.04161/) (2024)
- [Task-Adaptive Parameter-Efficient Fine-Tuning for Weather Foundation Models](https://mlsanity.com/paper/2509.22020/) (2025)
- [Bi-level Heterogeneous Learning for Time Series Foundation Models: A Federated Learning Approach](https://mlsanity.com/paper/2604.06727/) (2026)
- [M4Fog: A Global Multi-Regional, Multi-Modal, and Multi-Stage Dataset for Marine Fog Detection and Forecasting to Bridge Ocean and Atmosphere](https://mlsanity.com/paper/2406.13317/) (2024)
- [On the Regularization of Learnable Embeddings for Time Series Forecasting](https://mlsanity.com/paper/2410.14630/) (2024)
- [Spectral-Aware Text-to-Time Series Generation with Billion-Scale Multimodal Meteorological Data](https://mlsanity.com/paper/2603.27135/) (2026)
- [Heterogeneous Scientific Foundation Model Collaboration](https://mlsanity.com/paper/2604.27351/) (2026)
- [Less is more: Ensemble Learning for Retinal Disease Recognition Under Limited Resources](https://mlsanity.com/paper/2402.09747/) (2024)
- [VN-Net: Vision-Numerical Fusion Graph Convolutional Network for Sparse Spatio-Temporal Meteorological Forecasting](https://mlsanity.com/paper/2404.16037/) (2024)
- [Neural and Time-Series Approaches for Pricing Weather Derivatives: Performance and Regime Adaptation Using Satellite Data](https://mlsanity.com/paper/2411.12013/) (2024)
- [From Bias to Accountability: How the EU AI Act Confronts Challenges in European GeoAI Auditing](https://mlsanity.com/paper/2505.18236/) (2025)
- [Intelligence Foundation Model: A New Perspective to Approach Artificial General Intelligence](https://mlsanity.com/paper/2511.10119/) (2025)

## Abstract (arXiv)

As artificial intelligence (AI) continues to rapidly evolve, the realm of Earth and atmospheric sciences is increasingly adopting data-driven models, powered by progressive developments in deep learning (DL). Specifically, DL techniques are extensively utilized to decode the chaotic and nonlinear aspects of Earth systems, and to address climate challenges via understanding weather and climate data. Cutting-edge performance on specific tasks within narrower spatio-temporal scales has been achieved recently through DL. The rise of large models, specifically large language models (LLMs), has enabled fine-tuning processes that yield remarkable outcomes across various downstream tasks, thereby propelling the advancement of general AI. However, we are still navigating the initial stages of crafting general AI for weather and climate. In this survey, we offer an exhaustive, timely overview of state-of-the-art AI methodologies specifically engineered for weather and climate data, with a special focus on time series and text data. Our primary coverage encompasses four critical aspects: types of weather and climate data, principal model architectures, model scopes and applications, and datasets for weather and climate. Furthermore, in relation to the creation and application of foundation models for weather and climate data understanding, we delve into the field's prevailing challenges, offer crucial insights, and propose detailed avenues for future research. This comprehensive approach equips practitioners with the requisite knowledge to make substantial progress in this domain. Our survey encapsulates the most recent breakthroughs in research on large, data-driven models for weather and climate data understanding, emphasizing robust foundations, current advancements, practical applications, crucial resources, and prospective research opportunities.