# A deep learning approach to solar-irradiance forecasting in sky-videos

arXiv: https://arxiv.org/abs/1901.04881 · HTML: https://mlsanity.com/paper/1901.04881/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jan 2019.

## Built on by (most cited first, 10 of 10)

- [Benchmarking of Deep Learning Irradiance Forecasting Models from Sky Images -- an in-depth Analysis](https://mlsanity.com/paper/2102.00721/) (2021)
- [ECLIPSE : Envisioning CLoud Induced Perturbations in Solar Energy](https://mlsanity.com/paper/2104.12419/) (2021)
- [A Moment in the Sun: Solar Nowcasting from Multispectral Satellite Data using Self-Supervised Learning](https://mlsanity.com/paper/2112.13974/) (2021)
- [Sky Imager-Based Forecast of Solar Irradiance Using Machine Learning](https://mlsanity.com/paper/2310.17356/) (2023)
- [Solar Multimodal Transformer: Intraday Solar Irradiance Predictor using Public Cameras and Time Series](https://mlsanity.com/paper/2503.00250/) (2025)
- [Omnivision forecasting: combining satellite observations with sky images for improved intra-hour solar energy predictions](https://mlsanity.com/paper/2206.03207/) (2022)
- [Precise Forecasting of Sky Images Using Spatial Warping](https://mlsanity.com/paper/2409.12162/) (2024)
- [SPIRIT: Short-term Prediction of solar IRradIance for zero-shot Transfer learning using Foundation Models](https://mlsanity.com/paper/2502.10307/) (2025)
- [Tools for Extracting Spatio-Temporal Patterns in Meteorological Image Sequences: From Feature Engineering to Attention-Based Neural Networks](https://mlsanity.com/paper/2210.12310/) (2022)
- [Bidirectional Multimodal Fusion of Sky Images and Time-Series for Solar Forecasting with Large Language Models](https://mlsanity.com/paper/2609.11135/) (2026)

## Abstract (arXiv)

Ahead-of-time forecasting of incident solar-irradiance on a panel is indicative of expected energy yield and is essential for efficient grid distribution and planning. Traditionally, these forecasts are based on meteorological physics models whose parameters are tuned by coarse-grained radiometric tiles sensed from geo-satellites. This research presents a novel application of deep neural network approach to observe and estimate short-term weather effects from videos. Specifically, we use time-lapsed videos (sky-videos) obtained from upward facing wide-lensed cameras (sky-cameras) to directly estimate and forecast solar irradiance. We introduce and present results on two large publicly available datasets obtained from weather stations in two regions of North America using relatively inexpensive optical hardware. These datasets contain over a million images that span for 1 and 12 years respectively, the largest such collection to our knowledge. Compared to satellite based approaches, the proposed deep learning approach significantly reduces the normalized mean-absolute-percentage error for both nowcasting, i.e. prediction of the solar irradiance at the instance the frame is captured, as well as forecasting, ahead-of-time irradiance prediction for a duration for upto 4 hours.