# ST-MFNet Mini: Knowledge Distillation-Driven Frame Interpolation

arXiv: https://arxiv.org/abs/2302.08455 · HTML: https://mlsanity.com/paper/2302.08455/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation.
- Builds on 26 of its 33 references that are in the map.
- First posted on arXiv 16 Feb 2023.

## Placed in

- Knowledge distillation / [General knowledge distillation methods](https://mlsanity.com/distill/general-knowledge-distillation-methods/)

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [Softmax Splatting for Video Frame Interpolation](https://mlsanity.com/paper/2003.05534/) (2020)
- [Depth-Aware Video Frame Interpolation](https://mlsanity.com/paper/1904.00830/) (2019)
- [Video Frame Interpolation via Adaptive Separable Convolution](https://mlsanity.com/paper/1708.01692/) (2017)
- [FILM: Frame Interpolation for Large Motion](https://mlsanity.com/paper/2202.04901/) (2022)
- [Quadratic video interpolation](https://mlsanity.com/paper/1911.00627/) (2019)
- [AdaCoF: Adaptive Collaboration of Flows for Video Frame Interpolation](https://mlsanity.com/paper/1907.10244/) (2019)
- [XVFI: eXtreme Video Frame Interpolation](https://mlsanity.com/paper/2103.16206/) (2021)
- [Video Frame Interpolation Transformer](https://mlsanity.com/paper/2111.13817/) (2021)
- [BMBC:Bilateral Motion Estimation with Bilateral Cost Volume for Video Interpolation](https://mlsanity.com/paper/2007.12622/) (2020)
- [MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and Enhancement](https://mlsanity.com/paper/1810.08768/) (2018)
- [IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolation](https://mlsanity.com/paper/2205.14620/) (2022)
- [FLAVR: Flow-Agnostic Video Representations for Fast Frame Interpolation](https://mlsanity.com/paper/2012.08512/) (2020)
- [Multiple Video Frame Interpolation via Enhanced Deformable Separable Convolution](https://mlsanity.com/paper/2006.08070/) (2020)
- [Residual Conv-Deconv Grid Network for Semantic Segmentation](https://mlsanity.com/paper/1707.07958/) (2017)
- [CDFI: Compression-Driven Network Design for Frame Interpolation](https://mlsanity.com/paper/2103.10559/) (2021)
- [BVI-DVC: A Training Database for Deep Video Compression](https://mlsanity.com/paper/2003.13552/) (2020)
- [ST-MFNet: A Spatio-Temporal Multi-Flow Network for Frame Interpolation](https://mlsanity.com/paper/2111.15483/) (2021)
- [Orthant Based Proximal Stochastic Gradient Method for ℓ1-Regularized Optimization](https://mlsanity.com/paper/2004.03639/) (2020)
- [Neural Network Compression Via Sparse Optimization](https://mlsanity.com/paper/2011.04868/) (2020)
- [Enhancing Deformable Convolution based Video Frame Interpolation with Coarse-to-fine 3D CNN](https://mlsanity.com/paper/2202.07731/) (2022)

## Built on by (most cited first, 8 of 8)

- [LDMVFI: Video Frame Interpolation with Latent Diffusion Models](https://mlsanity.com/paper/2303.09508/) (2023)
- [MTKD: Multi-Teacher Knowledge Distillation for Image Super-Resolution](https://mlsanity.com/paper/2404.09571/) (2024)
- [Compressing Deep Image Super-resolution Models](https://mlsanity.com/paper/2401.00523/) (2023)
- [AIM 2024 Challenge on Efficient Video Super-Resolution for AV1 Compressed Content](https://mlsanity.com/paper/2409.17256/) (2024)
- [RTSR: A Real-Time Super-Resolution Model for AV1 Compressed Content](https://mlsanity.com/paper/2411.13362/) (2024)
- [RankDVQA-mini: Knowledge Distillation-Driven Deep Video Quality Assessment](https://mlsanity.com/paper/2312.08864/) (2023)
- [DaBiT: Depth and Blur informed Transformer for Video Focal Deblurring](https://mlsanity.com/paper/2407.01230/) (2024)
- [PocketDVDNet: Realtime Video Denoising for Real Camera Noise](https://mlsanity.com/paper/2601.16780/) (2026)

## Abstract (arXiv)

Currently, one of the major challenges in deep learning-based video frame interpolation (VFI) is the large model sizes and high computational complexity associated with many high performance VFI approaches. In this paper, we present a distillation-based two-stage workflow for obtaining compressed VFI models which perform competitively to the state of the arts, at a greatly reduced model size and complexity. Specifically, an optimisation-based network pruning method is first applied to a recently proposed frame interpolation model, ST-MFNet, which outperforms many other VFI methods but suffers from large model size. The resulting new network architecture achieves a 91% reduction in parameters and 35% increase in speed. Secondly, the performance of the new network is further enhanced through a teacher-student knowledge distillation training process using a Laplacian distillation loss. The final low complexity model, ST-MFNet Mini, achieves a comparable performance to most existing high-complex VFI methods, only outperformed by the original ST-MFNet. Our source code is available at https://github.com/crispianm/ST-MFNet-Mini