SkyReels-A1: Expressive Portrait Animation in Video Diffusion Transformers
On the map: Diffusion models → Audio, speech and music
Abstract (arXiv)
We present SkyReels-A1, a simple yet effective framework built upon video diffusion Transformer to facilitate portrait image animation. Existing methodologies still encounter issues, including identity distortion, background instability, and unrealistic facial dynamics, particularly in head-only animation scenarios. Besides, extending to accommodate diverse body proportions usually leads to visual inconsistencies or unnatural articulations. To address these challenges, SkyReels-A1 capitalizes on the strong generative capabilities of video DiT, enhancing facial motion transfer precision, identity retention, and temporal coherence. The system incorporates an expression-aware conditioning module that enables seamless video synthesis driven by expression-guided landmark inputs. Integrating the facial image-text alignment module strengthens the fusion of facial attributes with motion trajectories, reinforcing identity preservation. Additionally, SkyReels-A1 incorporates a multi-stage training paradigm to incrementally refine the correlation between expressions and motion while ensuring stable identity reproduction. Extensive empirical evaluations highlight the model's ability to produce visually coherent and compositionally diverse results, making it highly applicable to domains such as virtual avatars, remote communication, and digital media generation.
Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 20 of its 65 references that are in the map. First posted on arXiv 15 Feb 2025.
Builds on
20 of 65 references.
Closest ancestors · 3
Same direction, newest first
- 2024Loopy: Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependency4 Sep
- 2024FLUX that Plays Music1 Sep
- 2023From Pixels to Portraits: A Comprehensive Survey of Talking Head Generation Techniques and Applications30 Aug
Other references · 17
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
- 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 688
- 2024LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Controlcited by 206
- 2024EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditionscited by 196
- 2023SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Modelscited by 175
- 2024V-Express: Conditional Dropout for Progressive Training of Portrait Video Generationcited by 86
- 2024Scaling Diffusion Transformers to 16 Billion Parameterscited by 66
- 2024Scalable Diffusion Models with State Space Backbonecited by 45
- 2024Dimba: Transformer-Mamba Diffusion Modelscited by 26
5 more
- 2025Ingredients: Blending Custom Photos with Video Diffusion Transformerscited by 13
- 2023LatentWarp: Consistent Diffusion Latents for Zero-Shot Video-to-Video Translationcited by 12
- 2024MovieCharacter: A Tuning-Free Framework for Controllable Character Video Synthesiscited by 12
- 2024Video Diffusion Transformers are In-Context Learnerscited by 12
- 2024MotionCharacter: Fine-Grained Motion Controllable Human Video Generationcited by 1
Built on by
Citing papers per year: 2025: 23; 2026: 10.
33 papers.
Sort:
By area:
- 2025SkyReels-V2: Infinite-length Film Generative Modelcited by 229
- 2025Video World Models with Long-term Spatial Memorycited by 136
- 2025FantasyTalking: Realistic Talking Portrait Generation via Coherent Motion Synthesiscited by 77
- 2025SkyReels-A2: Compose Anything in Video Diffusion Transformerscited by 73
- 2025DreamActor-M1: Holistic, Expressive and Robust Human Image Animation with Hybrid Guidancecited by 36
- 2025OmniHuman-1.5: Instilling an Active Mind in Avatars via Cognitive Simulationcited by 32
- 2025SkyReels-Audio: Omni Audio-Conditioned Talking Portraits in Video Diffusion Transformerscited by 14
- 2025FantasyPortrait: Enhancing Multi-Character Portrait Animation with Expression-Augmented Diffusion Transformerscited by 13
- 2025KlingAvatar 2.0 Technical Reportcited by 13
- 2026SkyReels-V4: Multi-modal Video-Audio Generation, Inpainting and Editing modelcited by 12
- 2025TalkingHeadBench: A Multi-Modal Benchmark & Analysis of Talking-Head DeepFake Detectioncited by 11
- 2025PersonaLive! Expressive Portrait Image Animation for Live Streamingcited by 9
21 more
- 2025AlignHuman: Improving Motion and Fidelity via Timestep-Segment Preference Optimization for Audio-Driven Human Animationcited by 8
- 2025UniVid: Unifying Vision Tasks with Pre-trained Video Generation Modelscited by 5
- 2025FlashPortrait: 6x Faster Infinite Portrait Animation with Adaptive Latent Predictioncited by 5
- 2025DeX-Portrait: Disentangled and Expressive Portrait Animation via Explicit and Latent Motion Representationscited by 4
- 2026Kling-MotionControl Technical Reportcited by 4
- 2025Plan-X: Instruct Video Generation via Semantic Planningcited by 3
- 2026Image-to-Video Diffusion: From Foundations to Open Frontierscited by 3
- 2025FaceEditTalker: Controllable Talking Head Generation with Facial Attribute Editingcited by 1
- 2025FramePrompt: In-context Controllable Animation with Zero Structural Changescited by 1
- 2026Why Fake ? Unveiling the Semantic Vocabulary of Deepfake Detectorscited by 1
- 2025SwapAnyone: Consistent and Realistic Video Synthesis for Swapping Any Person into Any Videocited by 0
- 2025JWB-DH-V1: Benchmark for Joint Whole-Body Talking Avatar and Speech Generation Version 1cited by 0
- 2025Preview WB-DH: Towards Whole Body Digital Human Bench for the Generation of Whole-body Talking Avatar Videoscited by 0
- 2025YingVideo-MV: Music-Driven Multi-Stage Video Generationcited by 0
- 2025Instant Expressive Gaussian Head Avatars at Over 100 FPScited by 0
- 2026ExpPortrait: Expressive Portrait Generation via Personalized Representationcited by 0
- 2026Controllable Complex Human Motion Video Generation via Text-to-Skeleton Cascadescited by 0
- 2026UniBioTransfer: A Unified Framework for Multiple Biometrics Transfercited by 0
- 2026PerformRecast: Expression and Head Pose Disentanglement for Portrait Video Editingcited by 0
- 2026AsymTalker: Identity-Consistent Long-Term Talking Head Generation via Asymmetric Distillationcited by 0
- 2026Lumos-Nexus: Efficient Frequency Bridging with Homogeneous Latent Space for Video Unified Modelscited by 0
- 2026Why Fake ? Unveiling the Semantic Vocabulary of Deepfake Detectors8 Jul 2026
- 2026Lumos-Nexus: Efficient Frequency Bridging with Homogeneous Latent Space for Video Unified Models29 May 2026
- 2026Image-to-Video Diffusion: From Foundations to Open Frontiers17 May 2026
- 2026AsymTalker: Identity-Consistent Long-Term Talking Head Generation via Asymmetric Distillation1 May 2026
- 2026PerformRecast: Expression and Head Pose Disentanglement for Portrait Video Editing20 Mar 2026
- 2026UniBioTransfer: A Unified Framework for Multiple Biometrics Transfer20 Mar 2026
- 2026Controllable Complex Human Motion Video Generation via Text-to-Skeleton Cascades9 Mar 2026
- 2026Kling-MotionControl Technical Report3 Mar 2026
- 2026SkyReels-V4: Multi-modal Video-Audio Generation, Inpainting and Editing model25 Feb 2026
- 2026ExpPortrait: Expressive Portrait Generation via Personalized Representation23 Feb 2026
- 2025FlashPortrait: 6x Faster Infinite Portrait Animation with Adaptive Latent Prediction18 Dec 2025
- 2025Instant Expressive Gaussian Head Avatars at Over 100 FPS18 Dec 2025
21 more
- 2025DeX-Portrait: Disentangled and Expressive Portrait Animation via Explicit and Latent Motion Representations17 Dec 2025
- 2025KlingAvatar 2.0 Technical Report15 Dec 2025
- 2025PersonaLive! Expressive Portrait Image Animation for Live Streaming12 Dec 2025
- 2025YingVideo-MV: Music-Driven Multi-Stage Video Generation2 Dec 2025
- 2025Plan-X: Instruct Video Generation via Semantic Planning22 Nov 2025
- 2025UniVid: Unifying Vision Tasks with Pre-trained Video Generation Models26 Sep 2025
- 2025OmniHuman-1.5: Instilling an Active Mind in Avatars via Cognitive Simulation26 Aug 2025
- 2025Preview WB-DH: Towards Whole Body Digital Human Bench for the Generation of Whole-body Talking Avatar Videos12 Aug 2025
- 2025JWB-DH-V1: Benchmark for Joint Whole-Body Talking Avatar and Speech Generation Version 128 Jul 2025
- 2025FantasyPortrait: Enhancing Multi-Character Portrait Animation with Expression-Augmented Diffusion Transformers17 Jul 2025
- 2025FramePrompt: In-context Controllable Animation with Zero Structural Changes17 Jun 2025
- 2025AlignHuman: Improving Motion and Fidelity via Timestep-Segment Preference Optimization for Audio-Driven Human Animation11 Jun 2025
- 2025Video World Models with Long-term Spatial Memory5 Jun 2025
- 2025SkyReels-Audio: Omni Audio-Conditioned Talking Portraits in Video Diffusion Transformers1 Jun 2025
- 2025TalkingHeadBench: A Multi-Modal Benchmark & Analysis of Talking-Head DeepFake Detection30 May 2025
- 2025FaceEditTalker: Controllable Talking Head Generation with Facial Attribute Editing28 May 2025
- 2025SkyReels-V2: Infinite-length Film Generative Model17 Apr 2025
- 2025FantasyTalking: Realistic Talking Portrait Generation via Coherent Motion Synthesis7 Apr 2025
- 2025SkyReels-A2: Compose Anything in Video Diffusion Transformers3 Apr 2025
- 2025DreamActor-M1: Holistic, Expressive and Robust Human Image Animation with Hybrid Guidance2 Apr 2025
- 2025SwapAnyone: Consistent and Realistic Video Synthesis for Swapping Any Person into Any Video12 Mar 2025
- 2025SkyReels-A2: Compose Anything in Video Diffusion Transformerscited by 73
- 2025SkyReels-Audio: Omni Audio-Conditioned Talking Portraits in Video Diffusion Transformerscited by 14
- 2025FantasyPortrait: Enhancing Multi-Character Portrait Animation with Expression-Augmented Diffusion Transformerscited by 13
- 2025AlignHuman: Improving Motion and Fidelity via Timestep-Segment Preference Optimization for Audio-Driven Human Animationcited by 8
- 2025DeX-Portrait: Disentangled and Expressive Portrait Animation via Explicit and Latent Motion Representationscited by 4
- 2025Plan-X: Instruct Video Generation via Semantic Planningcited by 3
- 2026Image-to-Video Diffusion: From Foundations to Open Frontierscited by 3
- 2025FramePrompt: In-context Controllable Animation with Zero Structural Changescited by 1
- 2025JWB-DH-V1: Benchmark for Joint Whole-Body Talking Avatar and Speech Generation Version 1cited by 0
- 2025YingVideo-MV: Music-Driven Multi-Stage Video Generationcited by 0
- 2025Instant Expressive Gaussian Head Avatars at Over 100 FPScited by 0
- 2026ExpPortrait: Expressive Portrait Generation via Personalized Representationcited by 0
12 most cited of 13 All 13 (JSON)
- 2026Image-to-Video Diffusion: From Foundations to Open Frontiers17 May 2026
- 2026Controllable Complex Human Motion Video Generation via Text-to-Skeleton Cascades9 Mar 2026
- 2026ExpPortrait: Expressive Portrait Generation via Personalized Representation23 Feb 2026
- 2025Instant Expressive Gaussian Head Avatars at Over 100 FPS18 Dec 2025
- 2025DeX-Portrait: Disentangled and Expressive Portrait Animation via Explicit and Latent Motion Representations17 Dec 2025
- 2025YingVideo-MV: Music-Driven Multi-Stage Video Generation2 Dec 2025
- 2025Plan-X: Instruct Video Generation via Semantic Planning22 Nov 2025
- 2025JWB-DH-V1: Benchmark for Joint Whole-Body Talking Avatar and Speech Generation Version 128 Jul 2025
- 2025FantasyPortrait: Enhancing Multi-Character Portrait Animation with Expression-Augmented Diffusion Transformers17 Jul 2025
- 2025FramePrompt: In-context Controllable Animation with Zero Structural Changes17 Jun 2025
- 2025AlignHuman: Improving Motion and Fidelity via Timestep-Segment Preference Optimization for Audio-Driven Human Animation11 Jun 2025
- 2025SkyReels-Audio: Omni Audio-Conditioned Talking Portraits in Video Diffusion Transformers1 Jun 2025
12 newest of 13 All 13 (JSON)
- 2025SkyReels-V2: Infinite-length Film Generative Modelcited by 229
- 2025SkyReels-V2: Infinite-length Film Generative Model17 Apr 2025
- 2025Video World Models with Long-term Spatial Memorycited by 136
- 2025Video World Models with Long-term Spatial Memory5 Jun 2025
- 2025FantasyTalking: Realistic Talking Portrait Generation via Coherent Motion Synthesiscited by 77
- 2025DreamActor-M1: Holistic, Expressive and Robust Human Image Animation with Hybrid Guidancecited by 36
- 2025OmniHuman-1.5: Instilling an Active Mind in Avatars via Cognitive Simulationcited by 32
- 2025KlingAvatar 2.0 Technical Reportcited by 13
- 2026SkyReels-V4: Multi-modal Video-Audio Generation, Inpainting and Editing modelcited by 12
- 2025TalkingHeadBench: A Multi-Modal Benchmark & Analysis of Talking-Head DeepFake Detectioncited by 11
- 2025PersonaLive! Expressive Portrait Image Animation for Live Streamingcited by 9
- 2025UniVid: Unifying Vision Tasks with Pre-trained Video Generation Modelscited by 5
- 2025FlashPortrait: 6x Faster Infinite Portrait Animation with Adaptive Latent Predictioncited by 5
- 2026Kling-MotionControl Technical Reportcited by 4
- 2025FaceEditTalker: Controllable Talking Head Generation with Facial Attribute Editingcited by 1
- 2026Why Fake ? Unveiling the Semantic Vocabulary of Deepfake Detectorscited by 1
12 most cited of 18 All 18 (JSON)
- 2026Why Fake ? Unveiling the Semantic Vocabulary of Deepfake Detectors8 Jul 2026
- 2026Lumos-Nexus: Efficient Frequency Bridging with Homogeneous Latent Space for Video Unified Models29 May 2026
- 2026AsymTalker: Identity-Consistent Long-Term Talking Head Generation via Asymmetric Distillation1 May 2026
- 2026PerformRecast: Expression and Head Pose Disentanglement for Portrait Video Editing20 Mar 2026
- 2026UniBioTransfer: A Unified Framework for Multiple Biometrics Transfer20 Mar 2026
- 2026Kling-MotionControl Technical Report3 Mar 2026
- 2026SkyReels-V4: Multi-modal Video-Audio Generation, Inpainting and Editing model25 Feb 2026
- 2025FlashPortrait: 6x Faster Infinite Portrait Animation with Adaptive Latent Prediction18 Dec 2025
- 2025KlingAvatar 2.0 Technical Report15 Dec 2025
- 2025PersonaLive! Expressive Portrait Image Animation for Live Streaming12 Dec 2025
- 2025UniVid: Unifying Vision Tasks with Pre-trained Video Generation Models26 Sep 2025
- 2025OmniHuman-1.5: Instilling an Active Mind in Avatars via Cognitive Simulation26 Aug 2025
12 newest of 18 All 18 (JSON)
Similar, not cited
Close in wording, no citation either way
- 2025FactorPortrait: Controllable Portrait Animation via Disentangled Expression, Pose, and Viewpoint
- 2024Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer
- 2024X-Portrait: Expressive Portrait Animation with Hierarchical Motion Attention
- 2025Stable Video-Driven Portraits
- 2024Real-time One-Step Diffusion-based Expressive Portrait Videos Generation
- 2024Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animation
- 2025Hallo4: High-Fidelity Dynamic Portrait Animation via Direct Preference Optimization
- 2025MagicMirror: ID-Preserved Video Generation in Video Diffusion Transformers