# Learning for Unconstrained Space-Time Video Super-Resolution

arXiv: https://arxiv.org/abs/2102.13011 · HTML: https://mlsanity.com/paper/2102.13011/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 85 references that are in the map.
- First posted on arXiv 25 Feb 2021.

## Builds on (most cited first, 30 of 45)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [Learning Deep CNN Denoiser Prior for Image Restoration](https://mlsanity.com/paper/1704.03264/) (2017)
- [Accelerating the Super-Resolution Convolutional Neural Network](https://mlsanity.com/paper/1608.00367/) (2016)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [Depth-Aware Video Frame Interpolation](https://mlsanity.com/paper/1904.00830/) (2019)
- [GridDehazeNet: Attention-Based Multi-Scale Network for Image Dehazing](https://mlsanity.com/paper/1908.03245/) (2019)
- [TDAN: Temporally Deformable Alignment Network for Video Super-Resolution](https://mlsanity.com/paper/1812.02898/) (2018)
- [Deep Back-Projection Networks For Super-Resolution](https://mlsanity.com/paper/1803.02735/) (2018)
- [Video Frame Interpolation via Adaptive Separable Convolution](https://mlsanity.com/paper/1708.01692/) (2017)
- [Video Frame Synthesis using Deep Voxel Flow](https://mlsanity.com/paper/1702.02463/) (2017)
- [Meta-SR: A Magnification-Arbitrary Network for Super-Resolution](https://mlsanity.com/paper/1903.00875/) (2019)
- [Video Frame Interpolation via Adaptive Convolution](https://mlsanity.com/paper/1703.07514/) (2017)
- [GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution](https://mlsanity.com/paper/2012.00739/) (2020)
- [Context-aware Synthesis for Video Frame Interpolation](https://mlsanity.com/paper/1803.10967/) (2018)
- [Recurrent Back-Projection Network for Video Super-Resolution](https://mlsanity.com/paper/1903.10128/) (2019)
- [Quadratic video interpolation](https://mlsanity.com/paper/1911.00627/) (2019)

## Built on by (most cited first, 18 of 18)

- [Perception-Oriented Video Frame Interpolation via Asymmetric Blending](https://mlsanity.com/paper/2404.06692/) (2024)
- [AccFlow: Backward Accumulation for Long-Range Optical Flow](https://mlsanity.com/paper/2308.13133/) (2023)
- [ACDMSR: Accelerated Conditional Diffusion Models for Single Image Super-Resolution](https://mlsanity.com/paper/2307.00781/) (2023)
- [FastLLVE: Real-Time Low-Light Video Enhancement with Intensity-Aware Lookup Table](https://mlsanity.com/paper/2308.06749/) (2023)
- [GridDehazeNet+: An Enhanced Multi-Scale Network with Intra-Task Knowledge Transfer for Single Image Dehazing](https://mlsanity.com/paper/2103.13998/) (2021)
- [Light-VQA: A Multi-Dimensional Quality Assessment Model for Low-Light Video Enhancement](https://mlsanity.com/paper/2305.09512/) (2023)
- [TransMRSR: Transformer-based Self-Distilled Generative Prior for Brain MRI Super-Resolution](https://mlsanity.com/paper/2306.06669/) (2023)
- [End-to-end Transformer for Compressed Video Quality Enhancement](https://mlsanity.com/paper/2210.13827/) (2022)
- [NTIRE 2023 Quality Assessment of Video Enhancement Challenge](https://mlsanity.com/paper/2307.09729/) (2023)
- [Light-VQA+: A Video Quality Assessment Model for Exposure Correction with Vision-Language Guidance](https://mlsanity.com/paper/2405.03333/) (2024)
- [Text2QR: Harmonizing Aesthetic Customization and Scanning Robustness for Text-Guided QR Code Generation](https://mlsanity.com/paper/2403.06452/) (2024)
- [BONES: Near-Optimal Neural-Enhanced Video Streaming](https://mlsanity.com/paper/2310.09920/) (2023)
- [Space-Time Video Super-resolution with Neural Operator](https://mlsanity.com/paper/2404.06036/) (2024)
- [Enhancing Space-time Video Super-resolution via Spatial-temporal Feature Interaction](https://mlsanity.com/paper/2207.08960/) (2022)
- [A2-Edit: Precise Reference-Guided Image Editing of Arbitrary Objects and Ambiguous Masks](https://mlsanity.com/paper/2603.10685/) (2026)
- [Continuous Space-Time Video Super-Resolution Utilizing Long-Range Temporal Information](https://mlsanity.com/paper/2302.13256/) (2023)
- [Resolution-Agnostic Neural Compression for High-Fidelity Portrait Video Conferencing via Implicit Radiance Fields](https://mlsanity.com/paper/2402.16599/) (2024)
- [Compression-Realized Deep Structural Network for Video Quality Enhancement](https://mlsanity.com/paper/2405.06342/) (2024)

## Abstract (arXiv)

Recent years have seen considerable research activities devoted to video enhancement that simultaneously increases temporal frame rate and spatial resolution. However, the existing methods either fail to explore the intrinsic relationship between temporal and spatial information or lack flexibility in the choice of final temporal/spatial resolution. In this work, we propose an unconstrained space-time video super-resolution network, which can effectively exploit space-time correlation to boost performance. Moreover, it has complete freedom in adjusting the temporal frame rate and spatial resolution through the use of the optical flow technique and a generalized pixelshuffle operation. Our extensive experiments demonstrate that the proposed method not only outperforms the state-of-the-art, but also requires far fewer parameters and less running time.