FramePrompt: In-context Controllable Animation with Zero Structural Changes
On the map: Diffusion models → Video generation
Abstract (arXiv)
Generating controllable character animation from a reference image and motion guidance remains a challenging task due to the inherent difficulty of injecting appearance and motion cues into video diffusion models. Prior works often rely on complex architectures, explicit guider modules, or multi-stage processing pipelines, which increase structural overhead and hinder deployment. Inspired by the strong visual context modeling capacity of pre-trained video diffusion transformers, we propose FramePrompt, a minimalist yet powerful framework that treats reference images, skeleton-guided motion, and target video clips as a unified visual sequence. By reformulating animation as a conditional future prediction task, we bypass the need for guider networks and structural modifications. Experiments demonstrate that our method significantly outperforms representative baselines across various evaluation metrics while also simplifying training. Our findings highlight the effectiveness of sequence-level visual conditioning and demonstrate the potential of pre-trained models for controllable animation without architectural changes.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 39 of its 47 references that are in the map. First posted on arXiv 17 Jun 2025.
Builds on
39 of 47 references.
Closest ancestors · 16
Same direction, newest first
- 2025VACE: All-in-One Video Creation and Editing10 Mar
- 2025Animate Anyone 2: High-Fidelity Character Image Animation with Environment Affordance10 Feb
- 2025HumanDiT: Pose-Guided Diffusion Transformer for Long-form Human Motion Video Generation7 Feb
- 2024LTX-Video: Realtime Video Latent Diffusion30 Dec
- 2024Pyramidal Flow Matching for Efficient Video Generative Modeling8 Oct
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
- 2024UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animation3 Jun
- 2024Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance21 Mar
- 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation28 Nov
- 2023MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model27 Nov
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
- 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models7 Nov
4 more
Other references · 23
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
- 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
- 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
- 2020First Order Motion Model for Image Animationcited by 704
- 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 652
- 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
- 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 340
- 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 296
- 2021Motion Representations for Articulated Animationcited by 255
11 more
- 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 198
- 2024ControlNeXt: Powerful and Efficient Control for Image and Video Generationcited by 159
- 2023DisCo: Disentangled Control for Realistic Human Dance Generationcited by 142
- 2025OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Modelscited by 129
- 2024Animate-X: Universal Character Image Animation with Enhanced Motion Representationcited by 88
- 2024HumanVid: Demystifying Training Data for Camera-controllable Human Image Animationcited by 64
- 2024ACE: All-round Creator and Editor Following Instructions via Diffusion Transformercited by 56
- 2024RealisDance: Equip controllable character animation with realistic handscited by 37
- 2024DisPose: Disentangling Pose Guidance for Controllable Human Image Animationcited by 35
- 2025SkyReels-A1: Expressive Portrait Animation in Video Diffusion Transformerscited by 33
- 2025Lumina-OmniLV: A Unified Multimodal Framework for General Low-Level Visioncited by 14
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026DreamActor-M2: Universal Character Image Animation via Spatiotemporal In-Context Learning
- 2026SCAIL-2: Unifying Controlled Character Animation with End-to-End In-Context Conditioning
- 2023AnimateZero: Video Diffusion Models are Zero-Shot Image Animators
- 2024Controllable Longer Image Animation with Diffusion Models
- 2024Animate Your Motion: Turning Still Images into Dynamic Videos
- 2025Time-to-Move: Training-Free Motion Controlled Video Generation via Dual-Clock Denoising
- 2026MultiAnimate: Pose-Guided Image Animation Made Extensible
- 2025Generating Detailed Character Motion from Blocking Poses