# Deep Animation Video Interpolation in the Wild

arXiv: https://arxiv.org/abs/2104.02495 · HTML: https://mlsanity.com/paper/2104.02495/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 36 references that are in the map.
- First posted on arXiv 6 Apr 2021.

## Builds on (most cited first, 18 of 18)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [Softmax Splatting for Video Frame Interpolation](https://mlsanity.com/paper/2003.05534/) (2020)
- [Depth-Aware Video Frame Interpolation](https://mlsanity.com/paper/1904.00830/) (2019)
- [Video Frame Interpolation via Adaptive Separable Convolution](https://mlsanity.com/paper/1708.01692/) (2017)
- [Video Frame Synthesis using Deep Voxel Flow](https://mlsanity.com/paper/1702.02463/) (2017)
- [Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields](https://mlsanity.com/paper/1502.07411/) (2015)
- [Video Frame Interpolation via Adaptive Convolution](https://mlsanity.com/paper/1703.07514/) (2017)
- [Context-aware Synthesis for Video Frame Interpolation](https://mlsanity.com/paper/1803.10967/) (2018)
- [Quadratic video interpolation](https://mlsanity.com/paper/1911.00627/) (2019)
- [AdaCoF: Adaptive Collaboration of Flows for Video Frame Interpolation](https://mlsanity.com/paper/1907.10244/) (2019)
- [MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and Enhancement](https://mlsanity.com/paper/1810.08768/) (2018)
- [Residual Conv-Deconv Grid Network for Semantic Segmentation](https://mlsanity.com/paper/1707.07958/) (2017)
- [All at Once: Temporally Adaptive Multi-Frame Interpolation with Advanced Motion Modeling](https://mlsanity.com/paper/2007.11762/) (2020)
- [Towards the Automatic Anime Characters Creation with Generative Adversarial Networks](https://mlsanity.com/paper/1708.05509/) (2017)
- [Disentangling Style and Content in Anime Illustrations](https://mlsanity.com/paper/1905.10742/) (2019)

## Built on by (most cited first, 30 of 65)

- [AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolation](https://mlsanity.com/paper/2304.09790/) (2023)
- [IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolation](https://mlsanity.com/paper/2205.14620/) (2022)
- [LDMVFI: Video Frame Interpolation with Latent Diffusion Models](https://mlsanity.com/paper/2303.09508/) (2023)
- [ToonCrafter: Generative Cartoon Interpolation](https://mlsanity.com/paper/2405.17933/) (2024)
- [Many-to-many Splatting for Efficient Video Frame Interpolation](https://mlsanity.com/paper/2204.03513/) (2022)
- [A Unified Pyramid Recurrent Network for Video Frame Interpolation](https://mlsanity.com/paper/2211.03456/) (2022)
- [Sparse Global Matching for Video Frame Interpolation with Large Motion](https://mlsanity.com/paper/2404.06913/) (2024)
- [LVCD: Reference-based Lineart Video Colorization with Diffusion Models](https://mlsanity.com/paper/2409.12960/) (2024)
- [Generative Inbetweening through Frame-wise Conditions-Driven Video Generation](https://mlsanity.com/paper/2412.11755/) (2024)
- [AnimeSR: Learning Real-World Super-Resolution Models for Animation Videos](https://mlsanity.com/paper/2206.07038/) (2022)
- [AniDoc: Animation Creation Made Easier](https://mlsanity.com/paper/2412.14173/) (2024)
- [Improving the Perceptual Quality of 2D Animation Interpolation](https://mlsanity.com/paper/2111.12792/) (2021)
- [Splatting-based Synthesis for Video Frame Interpolation](https://mlsanity.com/paper/2201.10075/) (2022)
- [Frame Interpolation with Consecutive Brownian Bridge Diffusion](https://mlsanity.com/paper/2405.05953/) (2024)
- [EDEN: Enhanced Diffusion for High-quality Large-motion Video Frame Interpolation](https://mlsanity.com/paper/2503.15831/) (2025)
- [Erase Diffusion: Empowering Object Removal Through Calibrating Diffusion Pathways](https://mlsanity.com/paper/2503.07026/) (2025)
- [Exploring Motion Ambiguity and Alignment for High-Quality Video Frame Interpolation](https://mlsanity.com/paper/2203.10291/) (2022)
- [AniClipart: Clipart Animation with Text-to-Video Priors](https://mlsanity.com/paper/2404.12347/) (2024)
- [Sakuga-42M Dataset: Scaling Up Cartoon Research](https://mlsanity.com/paper/2405.07425/) (2024)
- [Generative AI for Cel-Animation: A Survey](https://mlsanity.com/paper/2501.06250/) (2025)
- [APISR: Anime Production Inspired Real-World Anime Super-Resolution](https://mlsanity.com/paper/2403.01598/) (2024)
- [Data-Efficient Unsupervised Interpolation Without Any Intermediate Frame for 4D Medical Images](https://mlsanity.com/paper/2404.01464/) (2024)
- [Generalizable Implicit Motion Modeling for Video Frame Interpolation](https://mlsanity.com/paper/2407.08680/) (2024)
- [Context-Aware Video Reconstruction for Rolling Shutter Cameras](https://mlsanity.com/paper/2205.12912/) (2022)
- [Learning Inclusion Matching for Animation Paint Bucket Colorization](https://mlsanity.com/paper/2403.18342/) (2024)
- [Motion-aware Latent Diffusion Models for Video Frame Interpolation](https://mlsanity.com/paper/2404.13534/) (2024)
- [BiM-VFI: Bidirectional Motion Field-Guided Frame Interpolation for Video with Non-uniform Motions](https://mlsanity.com/paper/2412.11365/) (2024)
- [Motion-from-Blur: 3D Shape and Motion Estimation of Motion-blurred Objects in Videos](https://mlsanity.com/paper/2111.14465/) (2021)
- [AnimeRun: 2D Animation Visual Correspondence from Open Source 3D Movies](https://mlsanity.com/paper/2211.05709/) (2022)
- [SportsSloMo: A New Benchmark and Baselines for Human-centric Video Frame Interpolation](https://mlsanity.com/paper/2308.16876/) (2023)

## Abstract (arXiv)

In the animation industry, cartoon videos are usually produced at low frame rate since hand drawing of such frames is costly and time-consuming. Therefore, it is desirable to develop computational models that can automatically interpolate the in-between animation frames. However, existing video interpolation methods fail to produce satisfying results on animation data. Compared to natural videos, animation videos possess two unique characteristics that make frame interpolation difficult: 1) cartoons comprise lines and smooth color pieces. The smooth areas lack textures and make it difficult to estimate accurate motions on animation videos. 2) cartoons express stories via exaggeration. Some of the motions are non-linear and extremely large. In this work, we formally define and study the animation video interpolation problem for the first time. To address the aforementioned challenges, we propose an effective framework, AnimeInterp, with two dedicated modules in a coarse-to-fine manner. Specifically, 1) Segment-Guided Matching resolves the "lack of textures" challenge by exploiting global matching among color pieces that are piece-wise coherent. 2) Recurrent Flow Refinement resolves the "non-linear and extremely large motion" challenge by recurrent predictions using a transformer-like architecture. To facilitate comprehensive training and evaluations, we build a large-scale animation triplet dataset, ATD-12K, which comprises 12,000 triplets with rich annotations. Extensive experiments demonstrate that our approach outperforms existing state-of-the-art interpolation methods for animation videos. Notably, AnimeInterp shows favorable perceptual quality and robustness for animation scenarios in the wild. The proposed dataset and code are available at https://github.com/lisiyao21/AnimeInterp/.