# FISR: Deep Joint Frame Interpolation and Super-Resolution with a Multi-scale Temporal Loss

arXiv: https://arxiv.org/abs/1912.07213 · HTML: https://mlsanity.com/paper/1912.07213/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Dec 2019.

## Built on by (most cited first, 28 of 28)

- [AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolation](https://mlsanity.com/paper/2304.09790/) (2023)
- [XVFI: eXtreme Video Frame Interpolation](https://mlsanity.com/paper/2103.16206/) (2021)
- [IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolation](https://mlsanity.com/paper/2205.14620/) (2022)
- [VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolution](https://mlsanity.com/paper/2206.04647/) (2022)
- [VEnhancer: Generative Space-Time Enhancement for Video Generation](https://mlsanity.com/paper/2407.07667/) (2024)
- [Temporal Modulation Network for Controllable Space-Time Video Super-Resolution](https://mlsanity.com/paper/2104.10642/) (2021)
- [Splatting-based Synthesis for Video Frame Interpolation](https://mlsanity.com/paper/2201.10075/) (2022)
- [DeMFI: Deep Joint Deblurring and Multi-Frame Interpolation with Flow-Guided Attentive Correlation and Recursive Boosting](https://mlsanity.com/paper/2111.09985/) (2021)
- [Spatial-Temporal Space Hand-in-Hand: Spatial-Temporal Video Super-Resolution via Cycle-Projected Mutual Learning](https://mlsanity.com/paper/2205.05264/) (2022)
- [Efficient Space-time Video Super Resolution using Low-Resolution Flow and Mask Upsampling](https://mlsanity.com/paper/2104.05778/) (2021)
- [AceVFI: A Comprehensive Survey of Advances in Video Frame Interpolation](https://mlsanity.com/paper/2506.01061/) (2025)
- [MEGAN: Memory Enhanced Graph Attention Network for Space-Time Video Super-Resolution](https://mlsanity.com/paper/2110.15327/) (2021)
- [Video Frame Interpolation with Densely Queried Bilateral Correlation](https://mlsanity.com/paper/2304.13596/) (2023)
- [Scale-Adaptive Feature Aggregation for Efficient Space-Time Video Super-Resolution](https://mlsanity.com/paper/2310.17294/) (2023)
- [HR-INR: Continuous Space-Time Video Super-Resolution via Event Camera](https://mlsanity.com/paper/2405.13389/) (2024)
- [Zooming SlowMo: An Efficient One-Stage Framework for Space-Time Video Super-Resolution](https://mlsanity.com/paper/2104.07473/) (2021)
- [Dual-Camera Smooth Zoom on Mobile Phones](https://mlsanity.com/paper/2404.04908/) (2024)
- [Ada-VSR: Adaptive Video Super-Resolution with Meta-Learning](https://mlsanity.com/paper/2108.02832/) (2021)
- [You Only Align Once: Bidirectional Interaction for Spatial-Temporal Video Super-Resolution](https://mlsanity.com/paper/2207.06345/) (2022)
- [Learning Spatio-Temporal Downsampling for Effective Video Upscaling](https://mlsanity.com/paper/2203.08140/) (2022)
- [Video Frame Interpolation with Transformer](https://mlsanity.com/paper/2205.07230/) (2022)
- [Enhancing Space-time Video Super-resolution via Spatial-temporal Feature Interaction](https://mlsanity.com/paper/2207.08960/) (2022)
- [Deep Learning Empowered Super-Resolution: A Comprehensive Survey and Future Prospects](https://mlsanity.com/paper/2509.22692/) (2025)
- [Towards Robust and Generalizable Continuous Space-Time Video Super-Resolution with Events](https://mlsanity.com/paper/2510.03833/) (2025)
- [Taming Real-World Space-Time Video Super-Resolution with One-Step Diffusion](https://mlsanity.com/paper/2601.20308/) (2026)
- [Uncertainty-Guided Spatial Pruning Architecture for Efficient Frame Interpolation](https://mlsanity.com/paper/2307.16555/) (2023)
- [Perceptual Refinement of an End-to-End Video Streaming Pipeline via Generative AI Layers](https://mlsanity.com/paper/2609.19215/) (2026)
- [Accurate Motion Estimation with B'ezier Control Point for Efficient Frame Interpolation](https://mlsanity.com/paper/2609.23408/) (2026)

## Abstract (arXiv)

Super-resolution (SR) has been widely used to convert low-resolution legacy videos to high-resolution (HR) ones, to suit the increasing resolution of displays (e.g. UHD TVs). However, it becomes easier for humans to notice motion artifacts (e.g. motion judder) in HR videos being rendered on larger-sized display devices. Thus, broadcasting standards support higher frame rates for UHD (Ultra High Definition) videos (4K@60 fps, 8K@120 fps), meaning that applying SR only is insufficient to produce genuine high quality videos. Hence, to up-convert legacy videos for realistic applications, not only SR but also video frame interpolation (VFI) is necessitated. In this paper, we first propose a joint VFI-SR framework for up-scaling the spatio-temporal resolution of videos from 2K 30 fps to 4K 60 fps. For this, we propose a novel training scheme with a multi-scale temporal loss that imposes temporal regularization on the input video sequence, which can be applied to any general video-related task. The proposed structure is analyzed in depth with extensive experiments.