Alignment is All You Need: A Training-free Augmentation Strategy for Pose-guided Video Generation
Not on the map. Why?
Abstract (arXiv)
Character animation is a transformative field in computer graphics and vision, enabling dynamic and realistic video animations from static images. Despite advancements, maintaining appearance consistency in animations remains a challenge. Our approach addresses this by introducing a training-free framework that ensures the generated video sequence preserves the reference image's subtleties, such as physique and proportions, through a dual alignment strategy. We decouple skeletal and motion priors from pose information, enabling precise control over animation generation. Our method also improves pixel-level alignment for conditional control from the reference character, enhancing the temporal consistency and visual cohesion of animations. Our method significantly enhances the quality of video generation without the need for large datasets or expensive computational resources.
Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 36 of its 43 references that are in the map. First posted on arXiv 29 Aug 2024.
Builds on
36 of 43 references.
References · 36
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
- 2022Video Diffusion Modelscited by 2,280
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
- 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
- 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
24 more
- 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
- 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
- 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
- 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
- 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 683
- 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
- 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
- 2018DensePose: Dense Human Pose Estimation In The Wildcited by 573
- 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
- 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 462
- 2023MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Modelcited by 330
- 2023Effective Whole-body Pose Estimation with Two-stages Distillationcited by 307
- 2023Composer: Creative and Controllable Image Synthesis with Composable Conditionscited by 303
- 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translationcited by 253
- 2023TryOnDiffusion: A Tale of Two UNetscited by 141
- 2023Advancing Pose-Guided Image Synthesis with Progressive Conditional Diffusion Modelscited by 104
- 2023Bridging Vision and Language Encoders: Parameter-Efficient Tuning for Referring Image Segmentationcited by 61
- 2024Coarse-to-Fine Latent Diffusion for Pose-Guided Person Image Synthesiscited by 48
- 2024MambaTalk: Efficient Holistic Gesture Synthesis with Selective State Space Modelscited by 46
- 2023G2L: Semantically Aligned and Uniform Video Grounding via Geodesic and Game Theorycited by 21
- 2023Chain of Generation: Multi-Modal Gesture Synthesis via Cascaded Conditional Controlcited by 18
- 2024Freetalker: Controllable Speech and Text-Driven Gesture Generation Based on Diffusion Models for Enhanced Speaker Naturalnesscited by 18
- 2023JOTR: 3D Joint Contrastive Learning with Transformers for Occluded Human Mesh Recoverycited by 12
- 2023Exploiting Auxiliary Caption for Video Groundingcited by 10
Built on by
Citing papers per year: 2024: 1; 2025: 8; 2026: 1.
10 papers.
- 2025Controllable Video Generation: A Surveycited by 93
- 2025HunyuanPortrait: Implicit Condition Control for Enhanced Portrait Animationcited by 55
- 2024DisPose: Disentangling Pose Guidance for Controllable Human Image Animationcited by 35
- 2025FireEdit: Fine-grained Instruction-based Image Editing via Region-aware Vision Language Modelcited by 31
- 2025Audio-visual Controlled Video Diffusion with Masked Selective State Spaces Modeling for Natural Talking Head Generationcited by 17
- 2025MVPortrait: Text-Guided Motion and Emotion Control for Multi-view Vivid Portrait Animationcited by 14
- 2026RoboStereo: Dual-Tower 4D Embodied World Models for Unified Policy Optimizationcited by 9
- 2025Zo3T: Zero-Shot 3D-Aware Trajectory-Guided Image-to-Video Generation via Test-Time Trainingcited by 7
- 2025Separate to Collaborate: Dual-Stream Diffusion Model for Coordinated Piano Hand Motion Synthesiscited by 4
- 2025InterAnimate: Taming Region-aware Diffusion Model for Realistic Human Interaction Animationcited by 4
- 2026RoboStereo: Dual-Tower 4D Embodied World Models for Unified Policy Optimization13 Mar 2026
- 2025Zo3T: Zero-Shot 3D-Aware Trajectory-Guided Image-to-Video Generation via Test-Time Training8 Sep 2025
- 2025Controllable Video Generation: A Survey22 Jul 2025
- 2025InterAnimate: Taming Region-aware Diffusion Model for Realistic Human Interaction Animation15 Apr 2025
- 2025Separate to Collaborate: Dual-Stream Diffusion Model for Coordinated Piano Hand Motion Synthesis14 Apr 2025
- 2025Audio-visual Controlled Video Diffusion with Masked Selective State Spaces Modeling for Natural Talking Head Generation3 Apr 2025
- 2025FireEdit: Fine-grained Instruction-based Image Editing via Region-aware Vision Language Model25 Mar 2025
- 2025MVPortrait: Text-Guided Motion and Emotion Control for Multi-view Vivid Portrait Animation25 Mar 2025
- 2025HunyuanPortrait: Implicit Condition Control for Enhanced Portrait Animation24 Mar 2025
- 2024DisPose: Disentangling Pose Guidance for Controllable Human Image Animation12 Dec 2024