# Framer: Interactive Frame Interpolation

arXiv: https://arxiv.org/abs/2410.18978 · HTML: https://mlsanity.com/paper/2410.18978/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 60 references that are in the map.
- First posted on arXiv 24 Oct 2024.

## Builds on (most cited first, 30 of 52)

- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [WorldSimBench: Towards Video Generation Models as World Simulators](https://mlsanity.com/paper/2410.18072/) (2024)
- [ModelScope Text-to-Video Technical Report](https://mlsanity.com/paper/2308.06571/) (2023)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [MotionCtrl: A Unified and Flexible Motion Controller for Video Generation](https://mlsanity.com/paper/2312.03641/) (2023)
- [VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models](https://mlsanity.com/paper/2401.09047/) (2024)
- [VideoCrafter1: Open Diffusion Models for High-Quality Video Generation](https://mlsanity.com/paper/2310.19512/) (2023)
- [CoTracker: It is Better to Track Together](https://mlsanity.com/paper/2307.07635/) (2023)
- [DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors](https://mlsanity.com/paper/2310.12190/) (2023)
- [VideoComposer: Compositional Video Synthesis with Motion Controllability](https://mlsanity.com/paper/2306.02018/) (2023)
- [CameraCtrl: Enabling Camera Control for Text-to-Video Generation](https://mlsanity.com/paper/2404.02101/) (2024)
- [OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation](https://mlsanity.com/paper/2407.02371/) (2024)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editing](https://mlsanity.com/paper/2306.14435/) (2023)
- [Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models](https://mlsanity.com/paper/2305.10474/) (2023)
- [Softmax Splatting for Video Frame Interpolation](https://mlsanity.com/paper/2003.05534/) (2020)
- [DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectory](https://mlsanity.com/paper/2308.08089/) (2023)
- [Depth-Aware Video Frame Interpolation](https://mlsanity.com/paper/1904.00830/) (2019)
- [Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold](https://mlsanity.com/paper/2305.10973/) (2023)
- [Video Frame Interpolation via Adaptive Separable Convolution](https://mlsanity.com/paper/1708.01692/) (2017)
- [AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolation](https://mlsanity.com/paper/2304.09790/) (2023)
- [SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Models](https://mlsanity.com/paper/2311.16933/) (2023)
- [DragAnything: Motion Control for Anything using Entity Representation](https://mlsanity.com/paper/2403.07420/) (2024)
- [VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control](https://mlsanity.com/paper/2407.12781/) (2024)
- [Context-aware Synthesis for Video Frame Interpolation](https://mlsanity.com/paper/1803.10967/) (2018)
- [FILM: Frame Interpolation for Large Motion](https://mlsanity.com/paper/2202.04901/) (2022)

## Built on by (most cited first, 24 of 24)

- [DICEPTION: A Generalist Diffusion Model for Visual Perceptual Tasks](https://mlsanity.com/paper/2502.17157/) (2025)
- [Generative Inbetweening through Frame-wise Conditions-Driven Video Generation](https://mlsanity.com/paper/2412.11755/) (2024)
- [Ming-Lite-Uni: Advancements in Unified Architecture for Natural Multimodal Interaction](https://mlsanity.com/paper/2505.02471/) (2025)
- [LayerAnimate: Layer-level Control for Animation](https://mlsanity.com/paper/2501.08295/) (2025)
- [AceVFI: A Comprehensive Survey of Advances in Video Frame Interpolation](https://mlsanity.com/paper/2506.01061/) (2025)
- [LongDWM: Cross-Granularity Distillation for Building a Long-Term Driving World Model](https://mlsanity.com/paper/2506.01546/) (2025)
- [Tinker: Diffusion's Gift to 3D--Multi-View Consistent Editing From Sparse Inputs without Per-Scene Optimization](https://mlsanity.com/paper/2508.14811/) (2025)
- [QuaDreamer: Controllable Panoramic Video Generation for Quadruped Robots](https://mlsanity.com/paper/2508.02512/) (2025)
- [In-2-4D: Inbetweening from Two Single-View Images to 4D Generation](https://mlsanity.com/paper/2504.08366/) (2025)
- [Motion Prior Distillation in Time Reversal Sampling for Generative Inbetweening](https://mlsanity.com/paper/2602.12679/) (2026)
- [EF-VI: Enhancing End-Frame Injection for Video Inbetweening](https://mlsanity.com/paper/2505.21205/) (2025)
- [EventDiff: A Unified and Efficient Diffusion Model Framework for Event-based Video Frame Interpolation](https://mlsanity.com/paper/2505.08235/) (2025)
- [FakeParts: a New Family of AI-Generated DeepFakes](https://mlsanity.com/paper/2508.21052/) (2025)
- [MultiCOIN: Multi-Modal COntrollable Video INbetweening](https://mlsanity.com/paper/2510.08561/) (2025)
- [SPLIT: Training-Free AI-Generated and Partially Edited Video Detection via Spatial Patch-Level Incoherence and Temporal Roughness](https://mlsanity.com/paper/2607.02886/) (2026)
- [Adapting Image-to-Video Diffusion Models for Large-Motion Frame Interpolation](https://mlsanity.com/paper/2412.17042/) (2024)
- [Semantic Frame Interpolation](https://mlsanity.com/paper/2507.05173/) (2025)
- [Preserving Source Video Realism: High-Fidelity Face Swapping for Cinematic Quality](https://mlsanity.com/paper/2512.07951/) (2025)
- [CineLOG: A Training Free Approach for Cinematic Long Video Generation](https://mlsanity.com/paper/2512.12209/) (2025)
- [Can Video Diffusion Models Predict Past Frames? Bidirectional Cycle Consistency for Reversible Interpolation](https://mlsanity.com/paper/2604.01700/) (2026)
- [How Animals Dance (When You're Not Looking)](https://mlsanity.com/paper/2505.23738/) (2025)
- [D3-Predictor: Noise-Free Deterministic Diffusion for Dense Prediction](https://mlsanity.com/paper/2512.07062/) (2025)
- [A Study of the Design Space of Motion and Disocclusion Control in Video Generation](https://mlsanity.com/paper/2512.13392/) (2025)
- [FC-VFI: Faithful and Consistent Video Frame Interpolation for High-FPS Slow Motion Video Generation](https://mlsanity.com/paper/2603.04899/) (2026)

## Abstract (arXiv)

We propose Framer for interactive frame interpolation, which targets producing smoothly transitioning frames between two images as per user creativity. Concretely, besides taking the start and end frames as inputs, our approach supports customizing the transition process by tailoring the trajectory of some selected keypoints. Such a design enjoys two clear benefits. First, incorporating human interaction mitigates the issue arising from numerous possibilities of transforming one image to another, and in turn enables finer control of local motions. Second, as the most basic form of interaction, keypoints help establish the correspondence across frames, enhancing the model to handle challenging cases (e.g., objects on the start and end frames are of different shapes and styles). It is noteworthy that our system also offers an "autopilot" mode, where we introduce a module to estimate the keypoints and refine the trajectory automatically, to simplify the usage in practice. Extensive experimental results demonstrate the appealing performance of Framer on various applications, such as image morphing, time-lapse video generation, cartoon interpolation, etc. The code, the model, and the interface will be released to facilitate further research.