# Open-Source Ground-based Sky Image Datasets for Very Short-term Solar Forecasting, Cloud Analysis and Modeling: A Comprehensive Survey

arXiv: https://arxiv.org/abs/2211.14709 · HTML: https://mlsanity.com/paper/2211.14709/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 248 references that are in the map.
- First posted on arXiv 27 Nov 2022.

## Builds on (most cited first, 30 of 31)

- [PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation](https://mlsanity.com/paper/1711.10871/) (2017)
- [Deep Photovoltaic Nowcasting](https://mlsanity.com/paper/1810.06327/) (2018)
- [Benchmarking of Deep Learning Irradiance Forecasting Models from Sky Images -- an in-depth Analysis](https://mlsanity.com/paper/2102.00721/) (2021)
- [CloudSegNet: A Deep Network for Nychthemeron Cloud Image Segmentation](https://mlsanity.com/paper/1904.07979/) (2019)
- [ECLIPSE : Envisioning CLoud Induced Perturbations in Solar Energy](https://mlsanity.com/paper/2104.12419/) (2021)
- [Color-based Segmentation of Sky/Cloud Images From Ground-based Cameras](https://mlsanity.com/paper/1606.03669/) (2016)
- [Convolutional Neural Networks applied to sky images for short-term solar irradiance forecasting](https://mlsanity.com/paper/2005.11246/) (2020)
- [SKIPP'D: a SKy Images and Photovoltaic Power Generation Dataset for Short-term Solar Forecasting](https://mlsanity.com/paper/2207.00913/) (2022)
- [Systematic study of color spaces and components for the segmentation of sky/cloud images](https://mlsanity.com/paper/1701.04520/) (2017)
- [Nighttime sky/cloud image segmentation](https://mlsanity.com/paper/1705.10583/) (2017)
- [Multi-label Cloud Segmentation Using a Deep Network](https://mlsanity.com/paper/1903.06562/) (2019)
- [SPIN: Simplifying Polar Invariance for Neural networks Application to vision-based irradiance forecasting](https://mlsanity.com/paper/2111.14507/) (2021)
- [High-Dynamic-Range Imaging for Cloud Segmentation](https://mlsanity.com/paper/1803.01071/) (2018)
- [Using GANs to Augment Data for Cloud Image Segmentation Task](https://mlsanity.com/paper/2106.03064/) (2021)
- [A Temporally Consistent Image-based Sun Tracking Algorithm for Solar Energy Forecasting Applications](https://mlsanity.com/paper/2012.01059/) (2020)
- [Why Unsupervised Deep Networks Generalize](https://mlsanity.com/paper/2012.03531/) (2020)
- [Girasol, a Sky Imaging and Global Solar Irradiance Dataset](https://mlsanity.com/paper/2102.07814/) (2021)
- [SkyCam: A Dataset of Sky Images and their Irradiance values](https://mlsanity.com/paper/2105.02922/) (2021)
- [Rough Set Based Color Channel Selection](https://mlsanity.com/paper/1611.00931/) (2016)
- [Detection of Clouds in Multiple Wind Velocity Fields using Ground-based Infrared Sky Images](https://mlsanity.com/paper/2105.03535/) (2021)
- [Short-term prediction of localized cloud motion using ground-based sky imagers](https://mlsanity.com/paper/1610.06666/) (2016)
- [A review on physical and data-driven based nowcasting methods using sky images](https://mlsanity.com/paper/2105.02959/) (2021)
- [ACLNet: An Attention and Clustering-based Cloud Segmentation Network](https://mlsanity.com/paper/2207.06277/) (2022)
- [Sky-image-based solar forecasting using deep learning with multi-location data: training models locally, globally or via transfer learning?](https://mlsanity.com/paper/2211.02108/) (2022)
- [Unsupervised Semantic-based Aggregation of Deep Convolutional Features](https://mlsanity.com/paper/1804.01422/) (2018)
- [Subjective Quality Assessment of Ground-based Camera Images](https://mlsanity.com/paper/1912.07192/) (2019)
- [Comparative Analysis of Methods for Cloud Segmentation in Ground-Based Infrared Images](https://mlsanity.com/paper/2012.06930/) (2020)
- [Explicit Basis Function Kernel Methods for Cloud Segmentation in Infrared Sky Images](https://mlsanity.com/paper/2102.06646/) (2021)
- [Segmentation Algorithms for Ground-Based Infrared Cloud Images](https://mlsanity.com/paper/2102.10151/) (2021)
- [Voting in Transfer Learning System for Ground-Based Cloud Classification](https://mlsanity.com/paper/2103.04667/) (2021)

## Built on by (most cited first, 2 of 2)

- [SkyGPT: Probabilistic Short-term Solar Forecasting Using Synthetic Sky Videos from Physics-constrained VideoGPT](https://mlsanity.com/paper/2306.11682/) (2023)
- [OpenConstruction: A Systematic Synthesis of Open Visual Datasets for Data-Centric Artificial Intelligence in Construction Monitoring](https://mlsanity.com/paper/2508.11482/) (2025)

## Abstract (arXiv)

Sky-image-based solar forecasting using deep learning has been recognized as a promising approach in reducing the uncertainty in solar power generation. However, one of the biggest challenges is the lack of massive and diversified sky image samples. In this study, we present a comprehensive survey of open-source ground-based sky image datasets for very short-term solar forecasting (i.e., forecasting horizon less than 30 minutes), as well as related research areas which can potentially help improve solar forecasting methods, including cloud segmentation, cloud classification and cloud motion prediction. We first identify 72 open-source sky image datasets that satisfy the needs of machine/deep learning. Then a database of information about various aspects of the identified datasets is constructed. To evaluate each surveyed datasets, we further develop a multi-criteria ranking system based on 8 dimensions of the datasets which could have important impacts on usage of the data. Finally, we provide insights on the usage of these datasets for different applications. We hope this paper can provide an overview for researchers who are looking for datasets for very short-term solar forecasting and related areas.