mlsanityData to 7 Oct 2026

EarthNet2021: A large-scale dataset and challenge for Earth surface forecasting as a guided video prediction task

arXiv 2104.10066 ↗First posted 16 Apr 2021cs.LG cs.AIcited by 34

Not on the map. Why?

Abstract (arXiv)

Satellite images are snapshots of the Earth surface. We propose to forecast them. We frame Earth surface forecasting as the task of predicting satellite imagery conditioned on future weather. EarthNet2021 is a large dataset suitable for training deep neural networks on the task. It contains Sentinel 2 satellite imagery at 20m resolution, matching topography and mesoscale (1.28km) meteorological variables packaged into 32000 samples. Additionally we frame EarthNet2021 as a challenge allowing for model intercomparison. Resulting forecasts will greatly improve (>x50) over the spatial resolution found in numerical models. This allows localized impacts from extreme weather to be predicted, thus supporting downstream applications such as crop yield prediction, forest health assessments or biodiversity monitoring. Find data, code, and how to participate at www.earthnet.tech

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 31 of its 74 references that are in the map. First posted on arXiv 16 Apr 2021.

Builds on

31 of 74 references.

References · 31

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  4. 2015Unsupervised Learning of Video Representations using LSTMscited by 434
  5. 2018Video-to-Video Synthesiscited by 388
  6. 2015Deep multi-scale video prediction beyond mean square errorcited by 297
  7. 2020WeatherBench: A benchmark dataset for data-driven weather forecastingcited by 272
  8. 2016Unsupervised Learning for Physical Interaction through Video Predictioncited by 256
  9. 2017Stochastic Variational Video Predictioncited by 234
  10. 2019BigEarthNet: A Large-Scale Benchmark Archive For Remote Sensing Image Understandingcited by 231
  11. 2016Deep Predictive Coding Networks for Video Prediction and Unsupervised Learningcited by 203
  12. 2018SpaceNet: A Remote Sensing Dataset and Challenge Seriescited by 178
19 more
  1. 2018Stochastic Adversarial Video Predictioncited by 173
  2. 2015Action-Conditional Video Prediction using Deep Networks in Atari Gamescited by 171
  3. 2017Unsupervised Learning of Disentangled Representations from Videocited by 138
  4. 2017Self-Supervised Visual Planning with Temporal Skip Connectionscited by 137
  5. 2016Video Pixel Networkscited by 108
  6. 2020Stochastic Latent Residual Video Predictioncited by 106
  7. 2020Improving data-driven global weather prediction using deep convolutional neural networks on a cubed spherecited by 95
  8. 2017DeepSD: Generating High Resolution Climate Change Projections through Single Image Super-Resolutioncited by 80
  9. 2014DenseNet: Implementing Efficient ConvNet Descriptor Pyramidscited by 66
  10. 2020Agriculture-Vision: A Large Aerial Image Database for Agricultural Pattern Analysiscited by 61
  11. 2018Disentangling Propagation and Generation for Video Predictioncited by 44
  12. 2020Assessing out-of-domain generalization for robust building damage detectioncited by 13
  13. 2020Utilizing UNet for the future traffic map prediction task Traffic4cast challenge 2020cited by 12
  14. 2020Physics-informed GANs for Coastal Flood Visualizationcited by 9
  15. 2020Estimating Crop Primary Productivity with Sentinel-2 and Landsat 8 using Machine Learning Methods Trained with Radiative Transfer Simulationscited by 7
  16. 2019Predicting Landscapes from Environmental Conditions Using Generative Networkscited by 5
  17. 2017PSIque: Next Sequence Prediction of Satellite Images using a Convolutional Sequence-to-Sequence Networkcited by 3
  18. 2019High Resolution Forecasting of Heat Waves impacts on Leaf Area Index by Multiscale Multitemporal Deep Learningcited by 3
  19. 2019Prediction of Soil Moisture Content Based On Satellite Data and Sequence-to-Sequence Networkscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 6; 2024: 9; 2025: 6; 2026: 8.

34 papers.

Sort:

By area:

  1. 2022DynamicEarthNet: Daily Multi-Spectral Satellite Dataset for Semantic Change Segmentationcited by 78
  2. 2023ClimateLearn: Benchmarking Machine Learning for Weather and Climate Modelingcited by 46
  3. 2022EarthNets: Empowering AI in Earth Observationcited by 40
  4. 2023Foundation Models for Weather and Climate Data Understanding: A Comprehensive Surveycited by 32
  5. 2023ClimateSet: A Large-Scale Climate Model Dataset for Machine Learningcited by 25
  6. 2022GLF-CR: SAR-Enhanced Cloud Removal with Global-Local Fusioncited by 24
  7. 2023Multi-modal learning for geospatial vegetation forecastingcited by 16
  8. 2023U-TILISE: A Sequence-to-sequence Model for Cloud Removal in Optical Satellite Time Seriescited by 8
  9. 2022Learning to forecast vegetation greenness at fine resolution over Africa with ConvLSTMscited by 7
  10. 2024Multi-scale Restoration of Missing Data in Optical Time-series Images with Masked Spatial-Temporal Attention Networkcited by 7
  11. 2024AllClear: A Comprehensive Dataset and Benchmark for Cloud Removal in Satellite Imagerycited by 7
  12. 2023Multi-Modal and Multi-Resolution Data Fusion for High-Resolution Cloud Removal: A Novel Baseline and Benchmarkcited by 6
22 more
  1. 2025A Diffusion-Based Framework for Terrain-Aware Remote Sensing Image Reconstructioncited by 6
  2. 2022Common Practices and Taxonomy in Deep Multi-view Fusion for Remote Sensing Applicationscited by 5
  3. 2024VegeDiff: Latent Diffusion Model for Geospatial Vegetation Forecastingcited by 4
  4. 2025ClimateBench-M: A Multi-Modal Climate Data Benchmark with a Simple Generative Methodcited by 4
  5. 2024DeepExtremeCubes: Integrating Earth system spatio-temporal data for impact assessment of climate extremescited by 3
  6. 2024Earth System Data Cubes: Avenues for advancing Earth system researchcited by 3
  7. 2024Explainable Earth Surface Forecasting under Extreme Eventscited by 2
  8. 2025An Analysis of Temporal Dropout in Earth Observation Time Series for Regression Taskscited by 2
  9. 2025Deep Space Weather Model: Long-Range Solar Flare Prediction from Multi-Wavelength Imagescited by 2
  10. 2025UniTS: Unified Spatio-Temporal Generative Model for Remote Sensingcited by 2
  11. 2024Implicit Neural Representations for Simultaneous Reduction and Continuous Reconstruction of Multi-Altitude Climate Datacited by 1
  12. 2024Identifying Spatio-Temporal Drivers of Extreme Eventscited by 1
  13. 2025Video Compression for Spatiotemporal Earth System Datacited by 1
  14. 2024Cluster-Segregate-Perturb (CSP): A Model-agnostic Explainability Pipeline for Spatiotemporal Land Surface Forecasting Modelscited by 0
  15. 2026Earth Science Foundation Models: From Perception to Reasoning and Discoverycited by 0
  16. 2026RealBench: Benchmarking Data-Driven Numerical Weather Forecasting Under Operational Conditions and Extreme Event Challengescited by 0
  17. 2026HLS-GPT: A Generative Pretrained Transformer (GPT) for Continental-Scale NASA Harmonized Landsat and Sentinel-2 (HLS) Reflectance Reconstruction Across All Bands on Arbitrary Datescited by 0
  18. 2026VegSim: A Geospatial World Model for Scenario-Conditioned Vegetation Simulationcited by 0
  19. 2026From Surface Forecasting to Observability Forecasting: A Latent World Model for Cloud-Aware EO Monitoringcited by 0
  20. 2026HeatCast: A Benchmark for Neighborhood-Scale LST Forecasting across 124 U.S. Citiescited by 0
  21. 2026Learning to Forecast Crop Growth from Earth Observation Datacited by 0
  22. 2026SPEAR NeXT Causal Latent Forecasting Across Multiple Horizons for Spectral Temporal Earth Representation Learningcited by 0