# VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolution

arXiv: https://arxiv.org/abs/2206.04647 · HTML: https://mlsanity.com/paper/2206.04647/

## Key facts

- Cited by 101 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 55 references that are in the map.
- First posted on arXiv 9 Jun 2022.

## Builds on (most cited first, 30 of 42)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond](https://mlsanity.com/paper/2012.02181/) (2020)
- [Softmax Splatting for Video Frame Interpolation](https://mlsanity.com/paper/2003.05534/) (2020)
- [Learning Shape Templates with Structured Implicit Functions](https://mlsanity.com/paper/1904.06447/) (2019)
- [Depth-Aware Video Frame Interpolation](https://mlsanity.com/paper/1904.00830/) (2019)
- [TDAN: Temporally Deformable Alignment Network for Video Super-Resolution](https://mlsanity.com/paper/1812.02898/) (2018)
- [Video Frame Interpolation via Adaptive Separable Convolution](https://mlsanity.com/paper/1708.01692/) (2017)
- [Flow-Guided Feature Aggregation for Video Object Detection](https://mlsanity.com/paper/1703.10025/) (2017)
- [Video Frame Interpolation via Adaptive Convolution](https://mlsanity.com/paper/1703.07514/) (2017)
- [Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation](https://mlsanity.com/paper/1611.05250/) (2016)
- [Context-aware Synthesis for Video Frame Interpolation](https://mlsanity.com/paper/1803.10967/) (2018)
- [NASA: Neural Articulated Shape Approximation](https://mlsanity.com/paper/1912.03207/) (2019)

## Built on by (most cited first, 30 of 101)

- [HiNeRV: Video Compression with Hierarchical Encoding-based Neural Representation](https://mlsanity.com/paper/2306.09818/) (2023)
- [VEnhancer: Generative Space-Time Enhancement for Video Generation](https://mlsanity.com/paper/2407.07667/) (2024)
- [FFNeRV: Flow-Guided Frame-Wise Neural Representations for Videos](https://mlsanity.com/paper/2212.12294/) (2022)
- [DNeRV: Modeling Inherent Dynamics via Difference Neural Representation for Videos](https://mlsanity.com/paper/2304.06544/) (2023)
- [Learning Spatial-Temporal Implicit Neural Representations for Event-Guided Video Super-Resolution](https://mlsanity.com/paper/2303.13767/) (2023)
- [Scalable Neural Video Representations with Learnable Positional Features](https://mlsanity.com/paper/2210.06823/) (2022)
- [CuNeRF: Cube-Based Neural Radiance Field for Zero-Shot Medical Image Arbitrary-Scale Super Resolution](https://mlsanity.com/paper/2303.16242/) (2023)
- [Sparse Global Matching for Video Frame Interpolation with Large Motion](https://mlsanity.com/paper/2404.06913/) (2024)
- [EDEN: Enhanced Diffusion for High-quality Large-motion Video Frame Interpolation](https://mlsanity.com/paper/2503.15831/) (2025)
- [AnyFlow: Arbitrary Scale Optical Flow with Implicit Neural Representation](https://mlsanity.com/paper/2303.16493/) (2023)
- [HIIF: Hierarchical Encoding based Implicit Image Function for Continuous Super-resolution](https://mlsanity.com/paper/2412.03748/) (2024)
- [DDMI: Domain-Agnostic Latent Diffusion Models for Synthesizing High-Quality Implicit Neural Representations](https://mlsanity.com/paper/2401.12517/) (2024)
- [Data-Efficient Unsupervised Interpolation Without Any Intermediate Frame for 4D Medical Images](https://mlsanity.com/paper/2404.01464/) (2024)
- [Generalizable Implicit Motion Modeling for Video Frame Interpolation](https://mlsanity.com/paper/2407.08680/) (2024)
- [LUVE : Latent-Cascaded Ultra-High-Resolution Video Generation with Dual Frequency Experts](https://mlsanity.com/paper/2602.11564/) (2026)
- [Knowledge-guided Machine Learning: Current Trends and Future Prospects](https://mlsanity.com/paper/2403.15989/) (2024)
- [FreSh: Frequency Shifting for Accelerated Neural Representation Learning](https://mlsanity.com/paper/2410.05050/) (2024)
- [SportsSloMo: A New Benchmark and Baselines for Human-centric Video Frame Interpolation](https://mlsanity.com/paper/2308.16876/) (2023)
- [Fast Encoding and Decoding for Implicit Video Representation](https://mlsanity.com/paper/2409.19429/) (2024)
- [UniINR: Event-guided Unified Rolling Shutter Correction, Deblurring, and Interpolation](https://mlsanity.com/paper/2305.15078/) (2023)
- [Arbitrary-Scale Video Super-Resolution with Structural and Textural Priors](https://mlsanity.com/paper/2407.09919/) (2024)
- [VeGaS: Video Gaussian Splatting](https://mlsanity.com/paper/2411.11024/) (2024)
- [SNeRV: Spectra-preserving Neural Representation for Video](https://mlsanity.com/paper/2501.01681/) (2025)
- [BF-STVSR: B-Splines and Fourier-Best Friends for High Fidelity Spatial-Temporal Video Super-Resolution](https://mlsanity.com/paper/2501.11043/) (2025)
- [Scale-Adaptive Feature Aggregation for Efficient Space-Time Video Super-Resolution](https://mlsanity.com/paper/2310.17294/) (2023)
- [Imaging Interiors: An Implicit Solution to Electromagnetic Inverse Scattering Problems](https://mlsanity.com/paper/2407.09352/) (2024)
- [Optimizing Rank for High-Fidelity Implicit Neural Representations](https://mlsanity.com/paper/2512.14366/) (2025)
- [A unified model for continuous conditional video prediction](https://mlsanity.com/paper/2210.05810/) (2022)
- [INR-V: A Continuous Representation Space for Video-based Generative Tasks](https://mlsanity.com/paper/2210.16579/) (2022)
- [Observation-Guided Meteorological Field Downscaling at Station Scale: A Benchmark and a New Method](https://mlsanity.com/paper/2401.11960/) (2024)

## Abstract (arXiv)

Videos typically record the streaming and continuous visual data as discrete consecutive frames. Since the storage cost is expensive for videos of high fidelity, most of them are stored in a relatively low resolution and frame rate. Recent works of Space-Time Video Super-Resolution (STVSR) are developed to incorporate temporal interpolation and spatial super-resolution in a unified framework. However, most of them only support a fixed up-sampling scale, which limits their flexibility and applications. In this work, instead of following the discrete representations, we propose Video Implicit Neural Representation (VideoINR), and we show its applications for STVSR. The learned implicit neural representation can be decoded to videos of arbitrary spatial resolution and frame rate. We show that VideoINR achieves competitive performances with state-of-the-art STVSR methods on common up-sampling scales and significantly outperforms prior works on continuous and out-of-training-distribution scales. Our project page is at http://zeyuan-chen.com/VideoINR/ .