mlsanityData to 7 Oct 2026

Foundation Models for Environmental Science: A Survey of Emerging Frontiers

arXiv 2504.04280 ↗First posted 5 Apr 2025cs.LG q-bio.QMcited by 4

Not on the map. Why?

Abstract (arXiv)

Modeling environmental ecosystems is essential for effective resource management, sustainable development, and understanding complex ecological processes. However, traditional data-driven methods face challenges in capturing inherently complex and interconnected processes and are further constrained by limited observational data in many environmental applications. Foundation models, which leverages large-scale pre-training and universal representations of complex and heterogeneous data, offer transformative opportunities for capturing spatiotemporal dynamics and dependencies in environmental processes, and facilitate adaptation to a broad range of applications. This survey presents a comprehensive overview of foundation model applications in environmental science, highlighting advancements in common environmental use cases including forward prediction, data generation, data assimilation, downscaling, inverse modeling, model ensembling, and decision-making across domains. We also detail the process of developing these models, covering data collection, architecture design, training, tuning, and evaluation. Through discussions on these emerging methods as well as their future opportunities, we aim to promote interdisciplinary collaboration that accelerates advancements in machine learning for driving scientific discovery in addressing critical environmental challenges.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 82 of its 235 references that are in the map. First posted on arXiv 5 Apr 2025.

Builds on

82 of 235 references.

References · 82

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  5. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  6. 2019A Comprehensive Survey on Transfer Learningcited by 1,154
  7. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  8. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
  9. 2020Learning Continuous Image Representation with Local Implicit Image Functioncited by 550
  10. 2022SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagerycited by 402
  11. 2022Multimodal Learning with Transformers: A Surveycited by 399
  12. 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 364
38 more
  1. 2017Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equationscited by 362
  2. 2023RemoteCLIP: A Vision Language Foundation Model for Remote Sensingcited by 357
  3. 2022Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learningcited by 302
  4. 2022GraphCast: Learning skillful medium-range global weather forecastingcited by 261
  5. 2023SpectralGPT: Spectral Remote Sensing Foundation Modelcited by 250
  6. 2018Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equationscited by 233
  7. 2023SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagerycited by 202
  8. 2020A review on outlier/anomaly detection in time series datacited by 187
  9. 2022Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecastcited by 164
  10. 2023BioCLIP: A Vision Foundation Model for the Tree of Lifecited by 159
  11. 2022Advancing Plain Vision Transformer Towards Remote Sensing Foundation Modelcited by 149
  12. 2023Foundation Models for Generalist Geospatial Artificial Intelligencecited by 143
  13. 2023DiffusionSat: A Generative Foundation Model for Satellite Imagerycited by 130
  14. 2023GEO-Bench: Toward Foundation Models for Earth Monitoringcited by 122
  15. 2022VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolutioncited by 101
  16. 2023A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4cited by 81
  17. 2023K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilizationcited by 69
  18. 2019Towards Learning Universal, Regional, and Local Hydrological Behaviors via Machine-Learning Applied to Large-Sample Datasetscited by 68
  19. 2020Physics-Guided Machine Learning for Scientific Discovery: An Application in Simulating Lake Temperature Profilescited by 59
  20. 2018Physics Guided RNNs for Modeling Dynamical Systems: A Case Study in Simulating Lake Temperature Profilescited by 55
  21. 2017Machine Learning for the Geosciences: Challenges and Opportunitiescited by 53
  22. 2020Deep Learning for Post-Processing Ensemble Weather Forecastscited by 52
  23. 2024Vision Foundation Models in Remote Sensing: A Surveycited by 52
  24. 2023SSL4EO-L: Datasets and Foundation Models for Landsat Imagerycited by 49
  25. 2024ClimateGPT: Towards AI Synthesizing Interdisciplinary Research on Climate Changecited by 46
  26. 2023OceanGPT: A Large Language Model for Ocean Science Taskscited by 42
  27. 2020Using machine learning to correct model error in data assimilation and forecast applicationscited by 40
  28. 2020Time Series Data Imputation: A Survey on Deep Learning Approachescited by 34
  29. 2023Large Language Models as Data Preprocessorscited by 34
  30. 2020A Survey on Transfer Learning in Natural Language Processingcited by 33
  31. 2023GeoGalactica: A Scientific Large Language Model in Geosciencecited by 32
  32. 2023Ecosystem Graphs: The Social Footprint of Foundation Modelscited by 31
  33. 2023Geo-knowledge-guided GPT models improve the extraction of location descriptions from disaster-related social media messagescited by 31
  34. 2024On the Foundations of Earth and Climate Foundation Modelscited by 31
  35. 2024FengWu-GHR: Learning the Kilometer-scale Medium-range Global Weather Forecastingcited by 30
  36. 2021Unsupervised Learning of Full-Waveform Inversion: Connecting CNN and Partial Differential Equation in a Loopcited by 29
  37. 2020From calibration to parameter learning: Harnessing the scaling effects of big data in geoscientific modelingcited by 26
  38. 2024PhilEO Bench: Evaluating Geo-Spatial Foundation Modelscited by 25

50 most cited of 82 All 82 (JSON)

Built on by

Cited by year

Citing papers per year: 2025: 4.

4 papers.