# Generative Inbetweening through Frame-wise Conditions-Driven Video Generation

arXiv: https://arxiv.org/abs/2412.11755 · HTML: https://mlsanity.com/paper/2412.11755/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 64 references that are in the map.
- First posted on arXiv 16 Dec 2024.

## Builds on (most cited first, 30 of 51)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation](https://mlsanity.com/paper/2311.17117/) (2023)
- [MotionCtrl: A Unified and Flexible Motion Controller for Video Generation](https://mlsanity.com/paper/2312.03641/) (2023)
- [DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors](https://mlsanity.com/paper/2310.12190/) (2023)
- [CameraCtrl: Enabling Camera Control for Text-to-Video Generation](https://mlsanity.com/paper/2404.02101/) (2024)
- [ControlVideo: Training-free Controllable Text-to-Video Generation](https://mlsanity.com/paper/2305.13077/) (2023)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [Effective Whole-body Pose Estimation with Two-stages Distillation](https://mlsanity.com/paper/2307.15880/) (2023)
- [MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolation](https://mlsanity.com/paper/2205.09853/) (2022)
- [DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editing](https://mlsanity.com/paper/2306.14435/) (2023)
- [Softmax Splatting for Video Frame Interpolation](https://mlsanity.com/paper/2003.05534/) (2020)
- [Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance](https://mlsanity.com/paper/2403.14781/) (2024)
- [Depth-Aware Video Frame Interpolation](https://mlsanity.com/paper/1904.00830/) (2019)
- [Video Frame Interpolation via Adaptive Separable Convolution](https://mlsanity.com/paper/1708.01692/) (2017)
- [Real-Time Intermediate Flow Estimation for Video Frame Interpolation](https://mlsanity.com/paper/2011.06294/) (2020)
- [Video Frame Synthesis using Deep Voxel Flow](https://mlsanity.com/paper/1702.02463/) (2017)
- [MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidance](https://mlsanity.com/paper/2406.19680/) (2024)
- [AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolation](https://mlsanity.com/paper/2304.09790/) (2023)
- [ControlNeXt: Powerful and Efficient Control for Image and Video Generation](https://mlsanity.com/paper/2408.06070/) (2024)
- [FILM: Frame Interpolation for Large Motion](https://mlsanity.com/paper/2202.04901/) (2022)

## Built on by (most cited first, 24 of 24)

- [Controllable Video Generation: A Survey](https://mlsanity.com/paper/2507.16869/) (2025)
- [Kontinuous Kontext: Continuous Strength Control for Instruction-based Image Editing](https://mlsanity.com/paper/2510.08532/) (2025)
- [OmniRoam: World Wandering via Long-Horizon Panoramic Video Generation](https://mlsanity.com/paper/2603.30045/) (2026)
- [Motion-Aware Generative Frame Interpolation](https://mlsanity.com/paper/2501.03699/) (2025)
- [AceVFI: A Comprehensive Survey of Advances in Video Frame Interpolation](https://mlsanity.com/paper/2506.01061/) (2025)
- [Motion Prior Distillation in Time Reversal Sampling for Generative Inbetweening](https://mlsanity.com/paper/2602.12679/) (2026)
- [EF-VI: Enhancing End-Frame Injection for Video Inbetweening](https://mlsanity.com/paper/2505.21205/) (2025)
- [Controllable Human-centric Keyframe Interpolation with Generative Prior](https://mlsanity.com/paper/2506.03119/) (2025)
- [Be Tangential to Manifold: Discovering Riemannian Metric for Diffusion Models](https://mlsanity.com/paper/2510.05509/) (2025)
- [SAGE: Structure-Aware Generative Video Transitions between Diverse Clips](https://mlsanity.com/paper/2510.24667/) (2025)
- [ColoDiff: Integrating Dynamic Consistency With Content Awareness for Colonoscopy Video Generation](https://mlsanity.com/paper/2602.23203/) (2026)
- [Anchoring and Rescaling Attention for Semantically Coherent Inbetweening](https://mlsanity.com/paper/2603.17651/) (2026)
- [Semantic Frame Interpolation](https://mlsanity.com/paper/2507.05173/) (2025)
- [Taming Real-World Space-Time Video Super-Resolution with One-Step Diffusion](https://mlsanity.com/paper/2601.20308/) (2026)
- [Can Video Diffusion Models Predict Past Frames? Bidirectional Cycle Consistency for Reversible Interpolation](https://mlsanity.com/paper/2604.01700/) (2026)
- [ActivityForensics: A Comprehensive Benchmark for Localizing Manipulated Activity in Videos](https://mlsanity.com/paper/2604.03819/) (2026)
- [Exploring the Role of Synthetic Data Augmentation in Controllable Human-Centric Video Generation](https://mlsanity.com/paper/2604.21291/) (2026)
- [FC-VFI: Faithful and Consistent Video Frame Interpolation for High-FPS Slow Motion Video Generation](https://mlsanity.com/paper/2603.04899/) (2026)
- [Ego-InBetween: Generating Object State Transitions in Ego-Centric Videos](https://mlsanity.com/paper/2604.17749/) (2026)
- [Generate Your Talking Avatar from Video Reference](https://mlsanity.com/paper/2604.27918/) (2026)
- [SVI360: Spherical Video Interpolation](https://mlsanity.com/paper/2607.11710/) (2026)
- [Bridging Event Streams and DiT: Event-Guided Video Frame Interpolation](https://mlsanity.com/paper/2608.10479/) (2026)
- [ZoomDiff: A High-Fidelity Diffusion Model for Dual-Camera Smooth Zooming](https://mlsanity.com/paper/2609.28083/) (2026)
- [Destination-Conditioned Adaptation with Semantic-Geometric Alignment for Generative Inbetweening](https://mlsanity.com/paper/2610.13187/) (2026)

## Abstract (arXiv)

Generative inbetweening aims to generate intermediate frame sequences by utilizing two key frames as input. Although remarkable progress has been made in video generation models, generative inbetweening still faces challenges in maintaining temporal stability due to the ambiguous interpolation path between two key frames. This issue becomes particularly severe when there is a large motion gap between input frames. In this paper, we propose a straightforward yet highly effective Frame-wise Conditions-driven Video Generation (FCVG) method that significantly enhances the temporal stability of interpolated video frames. Specifically, our FCVG provides an explicit condition for each frame, making it much easier to identify the interpolation path between two input frames and thus ensuring temporally stable production of visually plausible video frames. To achieve this, we suggest extracting matched lines from two input frames that can then be easily interpolated frame by frame, serving as frame-wise conditions seamlessly integrated into existing video generation models. In extensive evaluations covering diverse scenarios such as natural landscapes, complex human poses, camera movements and animations, existing methods often exhibit incoherent transitions across frames. In contrast, our FCVG demonstrates the capability to generate temporally stable videos using both linear and non-linear interpolation curves. Our project page and code are available at urlhttps://fcvg-inbetween.github.io/.