PFGNet: A Fully Convolutional Frequency-Guided Peripheral Gating Network for Efficient Spatiotemporal Predictive Learning
Not on the map. Why?
Abstract (arXiv)
Spatiotemporal predictive learning (STPL) aims to forecast future frames from past observations and is essential across a wide range of applications. Compared with recurrent or hybrid architectures, pure convolutional models offer superior efficiency and full parallelism, yet their fixed receptive fields limit their ability to adaptively capture spatially varying motion patterns. Inspired by biological center-surround organization and frequency-selective signal processing, we propose PFGNet, a fully convolutional framework that dynamically modulates receptive fields through pixel-wise frequency-guided gating. The core Peripheral Frequency Gating (PFG) block extracts localized spectral cues and adaptively fuses multi-scale large-kernel peripheral responses with learnable center suppression, effectively forming spatially adaptive band-pass filters. To maintain efficiency, all large kernels are decomposed into separable 1D convolutions (1 × k followed by k × 1), reducing per-channel computational cost from O(k2) to O(2k). PFGNet enables structure-aware spatiotemporal modeling without recurrence or attention. Experiments on Moving MNIST, TaxiBJ, Human3.6M, and KTH show that PFGNet delivers SOTA or near-SOTA forecasting performance with substantially fewer parameters and FLOPs. Our code is available at https://github.com/fhjdqaq/PFGNet.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 48 of its 72 references that are in the map. First posted on arXiv 24 Feb 2026.
Builds on
48 of 72 references.
References · 48
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
- 2022A ConvNet for the 2020scited by 3,545
- 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
- 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
- 2021Going deeper with Image Transformerscited by 759
- 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
- 2016Language Modeling with Gated Convolutional Networkscited by 688
- 2021MetaFormer Is Actually What You Need for Visioncited by 523
- 2015Unsupervised Learning of Video Representations using LSTMscited by 434
- 2016Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Predictioncited by 330
- 2016Dynamic Filter Networkscited by 318
36 more
- 2017Stochastic Variational Video Predictioncited by 234
- 2022Visual Attention Networkcited by 210
- 2016Deep Predictive Coding Networks for Video Prediction and Unsupervised Learningcited by 203
- 2020Disentangling Physical Dynamics from Unknown Factors for Unsupervised Video Predictioncited by 192
- 2022UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learningcited by 182
- 2021PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learningcited by 181
- 2022SimVP: Simpler yet Better Video Predictioncited by 181
- 2018Stochastic Adversarial Video Predictioncited by 173
- 2022More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsitycited by 142
- 2019Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolutioncited by 140
- 2018PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learningcited by 122
- 2016Video Pixel Networkscited by 108
- 2022HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutionscited by 103
- 2018Memory In Memory: A Predictive Neural Network for Learning Higher-Order Non-Stationarity from Spatiotemporal Dynamicscited by 91
- 2022Temporal Attention Unit: Towards Efficient Spatiotemporal Predictive Learningcited by 84
- 2023UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognitioncited by 64
- 2017Long-Term On-Board Prediction of People in Traffic Scenes under Uncertaintycited by 63
- 2021Video Prediction Recalling Long-term Motion Context via Memory Alignment Learningcited by 62
- 2022MogaNet: Multi-order Gated Aggregation Networkcited by 59
- 2023A Dynamic Multi-Scale Voxel Flow Network for Video Predictioncited by 53
- 2023OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learningcited by 53
- 2020Exploring Spatial-Temporal Multi-Frequency Analysis for High-Fidelity and Temporal-Consistency Video Predictioncited by 52
- 2017Folded Recurrent Neural Networks for Future Video Predictioncited by 44
- 2023PastNet: Introducing Physical Inductive Biases for Spatio-temporal Video Predictioncited by 44
- 2017RGB-D-based Human Motion Recognition with Deep Learning: A Surveycited by 37
- 2024VMRNN: Integrating Vision Mamba and LSTM for Efficient and Accurate Spatiotemporal Forecastingcited by 35
- 2023SwinLSTM:Improving Spatiotemporal Prediction Accuracy using Swin Transformer and LSTMcited by 32
- 2024PeLK: Parameter-efficient Large Kernel ConvNets with Peripheral Convolutioncited by 28
- 2022Peripheral Vision Transformercited by 27
- 2022SimVPv2: Towards Simple yet Powerful Spatiotemporal Predictive Learningcited by 20
- 2023MMVP: Motion-Matrix-based Video Predictioncited by 18
- 2023TFNet: Exploiting Temporal Cues for Fast and Accurate LiDAR Semantic Segmentationcited by 13
- 2024Scaling Up Your Kernels: Large Kernel Design in ConvNets towards Universal Representationscited by 8
- 2023Triplet Attention Transformer for Spatiotemporal Predictive Learningcited by 7
- 2024ShiftwiseConv: Small Convolutional Kernel with Large Kernel Effectcited by 4
- 2024STLight: a Fully Convolutional Approach for Efficient Predictive Learning by Spatio-Temporal joint Processingcited by 1
Built on by
Not cited yet.