# Video Super-Resolution Transformer with Masked Inter&Intra-Frame Attention

arXiv: https://arxiv.org/abs/2401.06312 · HTML: https://mlsanity.com/paper/2401.06312/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 43 references that are in the map.
- First posted on arXiv 12 Jan 2024.

## Builds on (most cited first, 30 of 33)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](https://mlsanity.com/paper/2104.13371/) (2021)
- [BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond](https://mlsanity.com/paper/2012.02181/) (2020)
- [TDAN: Temporally Deformable Alignment Network for Video Super-Resolution](https://mlsanity.com/paper/1812.02898/) (2018)
- [VRT: A Video Restoration Transformer](https://mlsanity.com/paper/2201.12288/) (2022)
- [Deep Feature Flow for Video Recognition](https://mlsanity.com/paper/1611.07715/) (2016)
- [Recurrent Video Restoration Transformer with Guided Deformable Attention](https://mlsanity.com/paper/2206.02146/) (2022)
- [Frame-Recurrent Video Super-Resolution](https://mlsanity.com/paper/1801.04590/) (2018)
- [Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation](https://mlsanity.com/paper/1611.05250/) (2016)
- [Detail-revealing Deep Video Super-resolution](https://mlsanity.com/paper/1704.02738/) (2017)
- [Video Super-Resolution Transformer](https://mlsanity.com/paper/2106.06847/) (2021)
- [Video Super-Resolution with Recurrent Structure-Detail Network](https://mlsanity.com/paper/2008.00455/) (2020)
- [MuCAN: Multi-Correspondence Aggregation Network for Video Super-Resolution](https://mlsanity.com/paper/2007.11803/) (2020)
- [Video Super-resolution with Temporal Group Attention](https://mlsanity.com/paper/2007.10595/) (2020)
- [Accel: A Corrective Fusion Network for Efficient Semantic Segmentation on Video](https://mlsanity.com/paper/1807.06667/) (2018)
- [Temporally Distributed Networks for Fast Video Semantic Segmentation](https://mlsanity.com/paper/2004.01800/) (2020)
- [Rethinking Alignment in Video Super-Resolution Transformers](https://mlsanity.com/paper/2207.08494/) (2022)
- [Efficient Video Super-Resolution through Recurrent Latent Space Propagation](https://mlsanity.com/paper/1909.08080/) (2019)
- [Low-Latency Video Semantic Segmentation](https://mlsanity.com/paper/1804.00389/) (2018)
- [Learning Trajectory-Aware Transformer for Video Super-Resolution](https://mlsanity.com/paper/2204.04216/) (2022)
- [Towards High Performance Video Object Detection](https://mlsanity.com/paper/1711.11577/) (2017)
- [Enhancing Video Super-Resolution via Implicit Resampling-based Alignment](https://mlsanity.com/paper/2305.00163/) (2023)
- [Skip-Convolutions for Efficient Video Processing](https://mlsanity.com/paper/2104.11487/) (2021)
- [Dynamic Kernel Distillation for Efficient Pose Estimation in Videos](https://mlsanity.com/paper/1908.09216/) (2019)
- [Learning Spatiotemporal Frequency-Transformer for Compressed Video Super-Resolution](https://mlsanity.com/paper/2208.03012/) (2022)

## Built on by (most cited first, 29 of 29)

- [VideoGigaGAN: Towards Detail-rich Video Super-Resolution](https://mlsanity.com/paper/2404.12388/) (2024)
- [PMQ-VE: Progressive Multi-Frame Quantization for Video Enhancement](https://mlsanity.com/paper/2505.12266/) (2025)
- [Stream-DiffVSR: Low-Latency Streamable Video Super-Resolution via Auto-Regressive Diffusion](https://mlsanity.com/paper/2512.23709/) (2025)
- [Event-Enhanced Blurry Video Super-Resolution](https://mlsanity.com/paper/2504.13042/) (2025)
- [VSRM: A Robust Mamba-Based Framework for Video Super-Resolution](https://mlsanity.com/paper/2506.22762/) (2025)
- [Blind Video Super-Resolution based on Implicit Kernels](https://mlsanity.com/paper/2503.07856/) (2025)
- [SpatioTemporal Difference Network for Video Depth Super-Resolution](https://mlsanity.com/paper/2508.01259/) (2025)
- [A Survey: Spatiotemporal Consistency in Video Generation](https://mlsanity.com/paper/2502.17863/) (2025)
- [Collaborative Feedback Discriminative Propagation for Video Super-Resolution](https://mlsanity.com/paper/2404.04745/) (2024)
- [Spatio-Temporal Distortion Aware Omnidirectional Video Super-Resolution](https://mlsanity.com/paper/2410.11506/) (2024)
- [FCVSR: A Frequency-aware Method for Compressed Video Super-Resolution](https://mlsanity.com/paper/2502.06431/) (2025)
- [Spatial Degradation-Aware and Temporal Consistent Diffusion Model for Compressed Video Super-Resolution](https://mlsanity.com/paper/2502.07381/) (2025)
- [Rethinking Diffusion Model-Based Video Super-Resolution: Leveraging Dense Guidance from Aligned Features](https://mlsanity.com/paper/2511.16928/) (2025)
- [Seeing the Unseen: Zooming in the Dark with Event Cameras](https://mlsanity.com/paper/2601.02206/) (2026)
- [Blind Restoration of High-Resolution Ultrasound Video](https://mlsanity.com/paper/2505.13915/) (2025)
- [A Systematic Investigation on Deep Learning-Based Omnidirectional Image and Video Super-Resolution](https://mlsanity.com/paper/2506.06710/) (2025)
- [Trajectory-aware Shifted State Space Models for Online Video Super-Resolution](https://mlsanity.com/paper/2508.10453/) (2025)
- [Deep Learning Empowered Super-Resolution: A Comprehensive Survey and Future Prospects](https://mlsanity.com/paper/2509.22692/) (2025)
- [Towards Robust and Generalizable Continuous Space-Time Video Super-Resolution with Events](https://mlsanity.com/paper/2510.03833/) (2025)
- [STCDiT: Spatio-Temporally Consistent Diffusion Transformer for High-Quality Video Super-Resolution](https://mlsanity.com/paper/2511.18786/) (2025)
- [HATIR: Heat-Aware Diffusion for Turbulent Infrared Video Super-Resolution](https://mlsanity.com/paper/2601.04682/) (2026)
- [Small Clips, Big Gains: Learning Long-Range Refocused Temporal Information for Video Super-Resolution](https://mlsanity.com/paper/2505.02159/) (2025)
- [FCA2: Frame Compression-Aware Autoencoder for Modular and Fast Compressed Video Super-Resolution](https://mlsanity.com/paper/2506.11545/) (2025)
- [MambaVSR: Content-Aware Scanning State Space Model for Video Super-Resolution](https://mlsanity.com/paper/2506.11768/) (2025)
- [APB-V: Accelerating Long-Video Understanding via Sequence-Parallelism-aware Approximate Attention](https://mlsanity.com/paper/2601.21444/) (2026)
- [Zero-Shot Video Restoration and Enhancement with Assistance of Video Diffusion Models](https://mlsanity.com/paper/2601.21922/) (2026)
- [Compressed-Domain-Aware Online Video Super-Resolution](https://mlsanity.com/paper/2603.07694/) (2026)
- [EvTexture++: Event-Driven Texture Enhancement for Video Super-Resolution](https://mlsanity.com/paper/2606.13580/) (2026)
- [Perceiving Better Moments: Cover Frame Reselection and Enhancement for Live Photos with the Live2K Dataset](https://mlsanity.com/paper/2607.04151/) (2026)

## Abstract (arXiv)

Recently, Vision Transformer has achieved great success in recovering missing details in low-resolution sequences, i.e., the video super-resolution (VSR) task. Despite its superiority in VSR accuracy, the heavy computational burden as well as the large memory footprint hinder the deployment of Transformer-based VSR models on constrained devices. In this paper, we address the above issue by proposing a novel feature-level masked processing framework: VSR with Masked Intra and inter frame Attention (MIA-VSR). The core of MIA-VSR is leveraging feature-level temporal continuity between adjacent frames to reduce redundant computations and make more rational use of previously enhanced SR features. Concretely, we propose an intra-frame and inter-frame attention block which takes the respective roles of past features and input features into consideration and only exploits previously enhanced features to provide supplementary information. In addition, an adaptive block-wise mask prediction module is developed to skip unimportant computations according to feature similarity between adjacent frames. We conduct detailed ablation studies to validate our contributions and compare the proposed method with recent state-of-the-art VSR approaches. The experimental results demonstrate that MIA-VSR improves the memory and computation efficiency over state-of-the-art methods, without trading off PSNR accuracy. The code is available at https://github.com/LabShuHangGU/MIA-VSR.