Solar Multimodal Transformer: Intraday Solar Irradiance Predictor using Public Cameras and Time Series
Not on the map. Why?
Abstract (arXiv)
Accurate intraday solar irradiance forecasting is crucial for optimizing dispatch planning and electricity trading. For this purpose, we introduce a novel and effective approach that includes three distinguishing components from the literature: 1) the uncommon use of single-frame public camera imagery; 2) solar irradiance time series scaled with a proposed normalization step, which boosts performance; and 3) a lightweight multimodal model, called Solar Multimodal Transformer (SMT), that delivers accurate short-term solar irradiance forecasting by combining images and scaled time series. Benchmarking against Solcast, a leading solar forecasting service provider, our model improved prediction accuracy by 25.95%. Our approach allows for easy adaptation to various camera specifications, offering broad applicability for real-world solar forecasting challenges.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 25 of its 44 references that are in the map. First posted on arXiv 28 Feb 2025.
Builds on
25 of 44 references.
References · 25
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
- 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
- 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,367
- 2017Non-local Neural Networkscited by 2,293
- 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
- 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
- 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
- 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,101
13 more
- 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,068
- 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 879
- 2020Transformer Interpretability Beyond Attention Visualizationcited by 627
- 2018Stacked Cross Attention for Image-Text Matchingcited by 416
- 2020Visual Transformers: Token-based Image Representation and Processing for Computer Visioncited by 334
- 2020Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentationcited by 326
- 2020Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformerscited by 288
- 2021LocalViT: Analyzing Locality in Vision Transformerscited by 228
- 2018BAM: Bottleneck Attention Modulecited by 172
- 2019Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directionscited by 172
- 2018Deep Photovoltaic Nowcastingcited by 21
- 2020Convolutional Neural Networks applied to sky images for short-term solar irradiance forecastingcited by 12
- 2019A deep learning approach to solar-irradiance forecasting in sky-videoscited by 10
Built on by
Citing papers per year: 2025: 2.
2 papers.