# Omniscient Video Super-Resolution

arXiv: https://arxiv.org/abs/2103.15683 · HTML: https://mlsanity.com/paper/2103.15683/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 33 references that are in the map.
- First posted on arXiv 29 Mar 2021.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond](https://mlsanity.com/paper/2012.02181/) (2020)
- [TDAN: Temporally Deformable Alignment Network for Video Super-Resolution](https://mlsanity.com/paper/1812.02898/) (2018)
- [Deep Back-Projection Networks For Super-Resolution](https://mlsanity.com/paper/1803.02735/) (2018)
- [Frame-Recurrent Video Super-Resolution](https://mlsanity.com/paper/1801.04590/) (2018)
- [Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation](https://mlsanity.com/paper/1611.05250/) (2016)
- [Detail-revealing Deep Video Super-resolution](https://mlsanity.com/paper/1704.02738/) (2017)
- [Recurrent Back-Projection Network for Video Super-Resolution](https://mlsanity.com/paper/1903.10128/) (2019)
- [Video Super-Resolution with Recurrent Structure-Detail Network](https://mlsanity.com/paper/2008.00455/) (2020)
- [Video Super-resolution with Temporal Group Attention](https://mlsanity.com/paper/2007.10595/) (2020)
- [Efficient Video Super-Resolution through Recurrent Latent Space Propagation](https://mlsanity.com/paper/1909.08080/) (2019)
- [Frame and Feature-Context Video Super-Resolution](https://mlsanity.com/paper/1909.13057/) (2019)

## Built on by (most cited first, 30 of 30)

- [VRT: A Video Restoration Transformer](https://mlsanity.com/paper/2201.12288/) (2022)
- [Recurrent Video Restoration Transformer with Guided Deformable Attention](https://mlsanity.com/paper/2206.02146/) (2022)
- [Learning Trajectory-Aware Transformer for Video Super-Resolution](https://mlsanity.com/paper/2204.04216/) (2022)
- [Learning Spatiotemporal Frequency-Transformer for Compressed Video Super-Resolution](https://mlsanity.com/paper/2208.03012/) (2022)
- [Learning Spatial Adaptation and Temporal Coherence in Diffusion Models for Video Super-Resolution](https://mlsanity.com/paper/2403.17000/) (2024)
- [Can SAM Boost Video Super-Resolution?](https://mlsanity.com/paper/2305.06524/) (2023)
- [Look Back and Forth: Video Super-Resolution with Explicit Temporal Difference Modeling](https://mlsanity.com/paper/2204.07114/) (2022)
- [Local-Global Temporal Difference Learning for Satellite Video Super-Resolution](https://mlsanity.com/paper/2304.04421/) (2023)
- [Temporal Consistency Learning of inter-frames for Video Super-Resolution](https://mlsanity.com/paper/2211.01639/) (2022)
- [Structured Sparsity Learning for Efficient Video Super-Resolution](https://mlsanity.com/paper/2206.07687/) (2022)
- [UltraVSR: Achieving Ultra-Realistic Video Super-Resolution with Efficient One-Step Diffusion Space](https://mlsanity.com/paper/2505.19958/) (2025)
- [Optical Flow Reusing for High-Efficiency Space-Time Video Super Resolution](https://mlsanity.com/paper/2110.06786/) (2021)
- [A Codec Information Assisted Framework for Efficient Compressed Video Super-Resolution](https://mlsanity.com/paper/2210.08229/) (2022)
- [Deep Blind Super-Resolution for Satellite Video](https://mlsanity.com/paper/2401.07139/) (2024)
- [A Survey of Deep Learning Video Super-Resolution](https://mlsanity.com/paper/2506.03216/) (2025)
- [Optical Flow for Video Super-Resolution: A Survey](https://mlsanity.com/paper/2203.10462/) (2022)
- [You Only Align Once: Bidirectional Interaction for Spatial-Temporal Video Super-Resolution](https://mlsanity.com/paper/2207.06345/) (2022)
- [Enhancing Quality for VVC Compressed Videos with Omniscient Quality Enhancement Model](https://mlsanity.com/paper/2504.19935/) (2025)
- [PP-MSVSR: Multi-Stage Video Super-Resolution](https://mlsanity.com/paper/2112.02828/) (2021)
- [Learning Spatiotemporal Frequency-Transformer for Low-Quality Video Super-Resolution](https://mlsanity.com/paper/2212.14046/) (2022)
- [ReBotNet: Fast Real-time Video Enhancement](https://mlsanity.com/paper/2303.13504/) (2023)
- [Cascaded Temporal Updating Network for Efficient Video Super-Resolution](https://mlsanity.com/paper/2408.14244/) (2024)
- [Implicit Neural Representation for Video Restoration](https://mlsanity.com/paper/2506.05488/) (2025)
- [Omnidirectional Video Super-Resolution using Deep Learning](https://mlsanity.com/paper/2506.14803/) (2025)
- [Deep Learning Empowered Super-Resolution: A Comprehensive Survey and Future Prospects](https://mlsanity.com/paper/2509.22692/) (2025)
- [Continuous Space-Time Video Super-Resolution Utilizing Long-Range Temporal Information](https://mlsanity.com/paper/2302.13256/) (2023)
- [A Lightweight Recurrent Grouping Attention Network for Video Super-Resolution](https://mlsanity.com/paper/2309.13940/) (2023)
- [An Empirical Study of Super-resolution on Low-resolution Micro-expression Recognition](https://mlsanity.com/paper/2310.10022/) (2023)
- [EPS: Efficient Patch Sampling for Video Overfitting in Deep Super-Resolution Model Training](https://mlsanity.com/paper/2411.16312/) (2024)
- [Implicit Neural Representation for Video and Image Super-Resolution](https://mlsanity.com/paper/2503.04665/) (2025)

## Abstract (arXiv)

Most recent video super-resolution (SR) methods either adopt an iterative manner to deal with low-resolution (LR) frames from a temporally sliding window, or leverage the previously estimated SR output to help reconstruct the current frame recurrently. A few studies try to combine these two structures to form a hybrid framework but have failed to give full play to it. In this paper, we propose an omniscient framework to not only utilize the preceding SR output, but also leverage the SR outputs from the present and future. The omniscient framework is more generic because the iterative, recurrent and hybrid frameworks can be regarded as its special cases. The proposed omniscient framework enables a generator to behave better than its counterparts under other frameworks. Abundant experiments on public datasets show that our method is superior to the state-of-the-art methods in objective metrics, subjective visual effects and complexity. Our code will be made public.