# PFGNet: A Fully Convolutional Frequency-Guided Peripheral Gating Network for Efficient Spatiotemporal Predictive Learning

arXiv: https://arxiv.org/abs/2602.20537 · HTML: https://mlsanity.com/paper/2602.20537/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 48 of its 72 references that are in the map.
- First posted on arXiv 24 Feb 2026.

## Builds on (most cited first, 30 of 48)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Going deeper with Image Transformers](https://mlsanity.com/paper/2103.17239/) (2021)
- [ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders](https://mlsanity.com/paper/2301.00808/) (2023)
- [Language Modeling with Gated Convolutional Networks](https://mlsanity.com/paper/1612.08083/) (2016)
- [MetaFormer Is Actually What You Need for Vision](https://mlsanity.com/paper/2111.11418/) (2021)
- [Unsupervised Learning of Video Representations using LSTMs](https://mlsanity.com/paper/1502.04681/) (2015)
- [Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction](https://mlsanity.com/paper/1610.00081/) (2016)
- [Dynamic Filter Networks](https://mlsanity.com/paper/1605.09673/) (2016)
- [Stochastic Variational Video Prediction](https://mlsanity.com/paper/1710.11252/) (2017)
- [Visual Attention Network](https://mlsanity.com/paper/2202.09741/) (2022)
- [Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning](https://mlsanity.com/paper/1605.08104/) (2016)
- [Disentangling Physical Dynamics from Unknown Factors for Unsupervised Video Prediction](https://mlsanity.com/paper/2003.01460/) (2020)
- [UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learning](https://mlsanity.com/paper/2201.04676/) (2022)
- [SimVP: Simpler yet Better Video Prediction](https://mlsanity.com/paper/2206.05099/) (2022)
- [PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning](https://mlsanity.com/paper/2103.09504/) (2021)
- [Stochastic Adversarial Video Prediction](https://mlsanity.com/paper/1804.01523/) (2018)
- [More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity](https://mlsanity.com/paper/2207.03620/) (2022)
- [Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution](https://mlsanity.com/paper/1904.05049/) (2019)
- [PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning](https://mlsanity.com/paper/1804.06300/) (2018)
- [Video Pixel Networks](https://mlsanity.com/paper/1610.00527/) (2016)
- [HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutions](https://mlsanity.com/paper/2207.14284/) (2022)
- [Memory In Memory: A Predictive Neural Network for Learning Higher-Order Non-Stationarity from Spatiotemporal Dynamics](https://mlsanity.com/paper/1811.07490/) (2018)
- [Temporal Attention Unit: Towards Efficient Spatiotemporal Predictive Learning](https://mlsanity.com/paper/2206.12126/) (2022)
- [UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition](https://mlsanity.com/paper/2311.15599/) (2023)
- [Long-Term On-Board Prediction of People in Traffic Scenes under Uncertainty](https://mlsanity.com/paper/1711.09026/) (2017)
- [Video Prediction Recalling Long-term Motion Context via Memory Alignment Learning](https://mlsanity.com/paper/2104.00924/) (2021)

## Abstract (arXiv)

Spatiotemporal predictive learning (STPL) aims to forecast future frames from past observations and is essential across a wide range of applications. Compared with recurrent or hybrid architectures, pure convolutional models offer superior efficiency and full parallelism, yet their fixed receptive fields limit their ability to adaptively capture spatially varying motion patterns. Inspired by biological center-surround organization and frequency-selective signal processing, we propose PFGNet, a fully convolutional framework that dynamically modulates receptive fields through pixel-wise frequency-guided gating. The core Peripheral Frequency Gating (PFG) block extracts localized spectral cues and adaptively fuses multi-scale large-kernel peripheral responses with learnable center suppression, effectively forming spatially adaptive band-pass filters. To maintain efficiency, all large kernels are decomposed into separable 1D convolutions (1 × k followed by k × 1), reducing per-channel computational cost from O(k2) to O(2k). PFGNet enables structure-aware spatiotemporal modeling without recurrence or attention. Experiments on Moving MNIST, TaxiBJ, Human3.6M, and KTH show that PFGNet delivers SOTA or near-SOTA forecasting performance with substantially fewer parameters and FLOPs. Our code is available at https://github.com/fhjdqaq/PFGNet.