mlsanityData to 7 Oct 2026

Foundation Models for Weather and Climate Data Understanding: A Comprehensive Survey

arXiv 2312.03014 ↗First posted 5 Dec 2023cs.LG cs.AI cs.CVcited by 32

Not on the map. Why?

Abstract (arXiv)

As artificial intelligence (AI) continues to rapidly evolve, the realm of Earth and atmospheric sciences is increasingly adopting data-driven models, powered by progressive developments in deep learning (DL). Specifically, DL techniques are extensively utilized to decode the chaotic and nonlinear aspects of Earth systems, and to address climate challenges via understanding weather and climate data. Cutting-edge performance on specific tasks within narrower spatio-temporal scales has been achieved recently through DL. The rise of large models, specifically large language models (LLMs), has enabled fine-tuning processes that yield remarkable outcomes across various downstream tasks, thereby propelling the advancement of general AI. However, we are still navigating the initial stages of crafting general AI for weather and climate. In this survey, we offer an exhaustive, timely overview of state-of-the-art AI methodologies specifically engineered for weather and climate data, with a special focus on time series and text data. Our primary coverage encompasses four critical aspects: types of weather and climate data, principal model architectures, model scopes and applications, and datasets for weather and climate. Furthermore, in relation to the creation and application of foundation models for weather and climate data understanding, we delve into the field's prevailing challenges, offer crucial insights, and propose detailed avenues for future research. This comprehensive approach equips practitioners with the requisite knowledge to make substantial progress in this domain. Our survey encapsulates the most recent breakthroughs in research on large, data-driven models for weather and climate data understanding, emphasizing robust foundations, current advancements, practical applications, crucial resources, and prospective research opportunities.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 164 of its 345 references that are in the map. First posted on arXiv 5 Dec 2023.

Builds on

164 of 345 references.

References · 164

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2023GPT-4 Technical Reportcited by 18,132
  8. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  9. 2022Training language models to follow instructions with human feedbackcited by 15,712
  10. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  11. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  12. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
38 more
  1. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  2. 2020Denoising Diffusion Implicit Modelscited by 9,655
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  4. 2023Segment Anythingcited by 8,158
  5. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  7. 2022Generative Adversarial Networkscited by 5,603
  8. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  9. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  10. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  11. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  12. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  13. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  14. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  15. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  16. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  17. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  18. 2020Reformer: The Efficient Transformercited by 1,658
  19. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
  20. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
  21. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  22. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  23. 2018ESRGAN: Enhanced Super-Resolution Generative Adversarial Networkscited by 1,250
  24. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
  25. 2017Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecastingcited by 1,125
  26. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
  27. 2022Diffusion Models in Vision: A Surveycited by 1,008
  28. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  29. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  30. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  31. 2018Hierarchical Graph Representation Learning with Differentiable Poolingcited by 786
  32. 2022FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operatorscited by 748
  33. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  34. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
  35. 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 560
  36. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 510
  37. 2022Transformers in Time Series: A Surveycited by 489
  38. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428

50 most cited of 164 All 164 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 18; 2025: 8; 2026: 6.

32 papers.

Sort:

By area:

  1. 2024Evaluating Large Language Models on Time Series Feature Understanding: A Comprehensive Taxonomy and Benchmarkcited by 38
  2. 2024On the Foundations of Earth and Climate Foundation Modelscited by 31
  3. 2024Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Modelscited by 23
  4. 2024WeatherQA: Can Multimodal Language Models Reason about Severe Weather?cited by 21
  5. 2024WildfireGPT: Tailored Large Language Model for Wildfire Analysiscited by 15
  6. 2024Personalized Adapter for Large Meteorology Model on Devices: Towards Weather Foundation Modelscited by 15
  7. 2025Deep Learning and Foundation Models for Weather Prediction: A Surveycited by 15
  8. 2024Federated Foundation Models on Heterogeneous Time Seriescited by 13
  9. 2026Foundation Models in Remote Sensing: Evolving from Unimodality to Multimodalitycited by 12
  10. 2025Satellite Observations Guided Diffusion Model for Accurate Meteorological States at Arbitrary Resolutioncited by 9
  11. 2025Temporal Neural Operator for Modeling Time-Dependent Physical Phenomenacited by 8
  12. 2024Observation-Guided Meteorological Field Downscaling at Station Scale: A Benchmark and a New Methodcited by 7
20 more
  1. 2024Evaluating and Benchmarking Foundation Models for Earth Observation and Geospatial AIcited by 7
  2. 2024State-observation augmented diffusion model for nonlinear assimilation with unknown dynamicscited by 7
  3. 2025A RAG-Based Multi-Agent LLM System for Natural Hazard Resilience and Adaptationcited by 5
  4. 2025SciVid: Cross-Domain Evaluation of Video Models in Scientific Applicationscited by 5
  5. 2024Interpretable Machine Learning for Weather and Climate Prediction: A Surveycited by 4
  6. 2024Federated Distillation for Medical Image Classification: Towards Trustworthy Computer-Aided Diagnosiscited by 4
  7. 2024Decoding complexity: how machine learning is redefining scientific discoverycited by 3
  8. 2025Task-Adaptive Parameter-Efficient Fine-Tuning for Weather Foundation Modelscited by 3
  9. 2026Bi-level Heterogeneous Learning for Time Series Foundation Models: A Federated Learning Approachcited by 2
  10. 2024M4Fog: A Global Multi-Regional, Multi-Modal, and Multi-Stage Dataset for Marine Fog Detection and Forecasting to Bridge Ocean and Atmospherecited by 1
  11. 2024On the Regularization of Learnable Embeddings for Time Series Forecastingcited by 1
  12. 2026Spectral-Aware Text-to-Time Series Generation with Billion-Scale Multimodal Meteorological Datacited by 1
  13. 2024Less is more: Ensemble Learning for Retinal Disease Recognition Under Limited Resourcescited by 0
  14. 2024VN-Net: Vision-Numerical Fusion Graph Convolutional Network for Sparse Spatio-Temporal Meteorological Forecastingcited by 0
  15. 2024Neural and Time-Series Approaches for Pricing Weather Derivatives: Performance and Regime Adaptation Using Satellite Datacited by 0
  16. 2025From Bias to Accountability: How the EU AI Act Confronts Challenges in European GeoAI Auditingcited by 0
  17. 2025Intelligence Foundation Model: A New Perspective to Approach Artificial General Intelligencecited by 0
  18. 2026Optimizing Multi-Agent Weather Captioning via Text Gradient Descent: A Training-Free Approach with Consensus-Aware Gradient Fusioncited by 0
  19. 2026Federated Weather Modeling on Sensor Datacited by 0
  20. 2026Deep Academic Survey: Stateful Agentic Closed-Loop Paradigm for Academic Survey Automationcited by 0