WaveSFNet: A Wavelet-Based Codec and Spatial--Frequency Dual-Domain Gating Network for Spatiotemporal Prediction
Not on the map. Why?
Abstract (arXiv)
Spatiotemporal predictive learning aims to forecast future frames from historical observations in an unsupervised manner, and is critical to a wide range of applications. The key challenge is to model long-range dynamics while preserving high-frequency details for sharp multi-step predictions. Existing efficient recurrent-free frameworks typically rely on strided convolutions or pooling for sampling, which tends to discard textures and boundaries, while purely spatial operators often struggle to balance local interactions with global propagation. To address these issues, we propose WaveSFNet, an efficient framework that unifies a wavelet-based codec with a spatial--frequency dual-domain gated spatiotemporal translator. The wavelet-based codec preserves high-frequency subband cues during downsampling and reconstruction. Meanwhile, the translator first injects adjacent-frame differences to explicitly enhance dynamic information, and then performs dual-domain gated fusion between large-kernel spatial local modeling and frequency-domain global modulation, together with gated channel interaction for cross-channel feature exchange. Extensive experiments demonstrate that WaveSFNet achieves competitive prediction accuracy on Moving MNIST, TaxiBJ, and WeatherBench, while maintaining low computational complexity. Our code is available at https://github.com/fhjdqaq/WaveSFNet.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 35 of its 45 references that are in the map. First posted on arXiv 24 Mar 2026.
Builds on
35 of 45 references.
References · 35
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2017Non-local Neural Networkscited by 2,290
- 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
- 2021Going deeper with Image Transformerscited by 759
- 2022FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operatorscited by 748
- 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
- 2021MetaFormer Is Actually What You Need for Visioncited by 523
- 2015Unsupervised Learning of Video Representations using LSTMscited by 434
- 2021FNet: Mixing Tokens with Fourier Transformscited by 343
- 2016Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Predictioncited by 330
- 2020WeatherBench: A benchmark dataset for data-driven weather forecastingcited by 272
- 2021Global Filter Networks for Image Classificationcited by 257
23 more
- 2022Patches Are All You Need?cited by 201
- 2020Disentangling Physical Dynamics from Unknown Factors for Unsupervised Video Predictioncited by 192
- 2018Multi-level Wavelet-CNN for Image Restorationcited by 184
- 2021PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learningcited by 181
- 2022SimVP: Simpler yet Better Video Predictioncited by 181
- 2022Earthformer: Exploring Space-Time Transformers for Earth System Forecastingcited by 146
- 2018PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learningcited by 122
- 2022HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutionscited by 103
- 2018Memory In Memory: A Predictive Neural Network for Learning Higher-Order Non-Stationarity from Spatiotemporal Dynamicscited by 91
- 2022Temporal Attention Unit: Towards Efficient Spatiotemporal Predictive Learningcited by 84
- 2022Wave-ViT: Unifying Wavelet and Transformers for Visual Representation Learningcited by 78
- 2017Long-Term On-Board Prediction of People in Traffic Scenes under Uncertaintycited by 63
- 2020Wavelet Integrated CNNs for Noise-Robust Image Classificationcited by 60
- 2022MogaNet: Multi-order Gated Aggregation Networkcited by 59
- 2023OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learningcited by 53
- 2023SpectFormer: Frequency and Attention is what you need in a Vision Transformercited by 48
- 2017Folded Recurrent Neural Networks for Future Video Predictioncited by 44
- 2023PastNet: Introducing Physical Inductive Biases for Spatio-temporal Video Predictioncited by 44
- 2024VMRNN: Integrating Vision Mamba and LSTM for Efficient and Accurate Spatiotemporal Forecastingcited by 35
- 2023SwinLSTM:Improving Spatiotemporal Prediction Accuracy using Swin Transformer and LSTMcited by 32
- 2022SimVPv2: Towards Simple yet Powerful Spatiotemporal Predictive Learningcited by 20
- 2023MMVP: Motion-Matrix-based Video Predictioncited by 18
- 2023Triplet Attention Transformer for Spatiotemporal Predictive Learningcited by 7
Built on by
Not cited yet.