mlsanityData to 7 Oct 2026

Alignment is All You Need: A Training-free Augmentation Strategy for Pose-guided Video Generation

arXiv 2408.16506 ↗First posted 29 Aug 2024cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

Character animation is a transformative field in computer graphics and vision, enabling dynamic and realistic video animations from static images. Despite advancements, maintaining appearance consistency in animations remains a challenge. Our approach addresses this by introducing a training-free framework that ensures the generated video sequence preserves the reference image's subtleties, such as physique and proportions, through a dual alignment strategy. We decouple skeletal and motion priors from pose information, enabling precise control over animation generation. Our method also improves pixel-level alignment for conditional control from the reference character, enhancing the temporal consistency and visual cohesion of animations. Our method significantly enhances the quality of video generation without the need for large datasets or expensive computational resources.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 36 of its 43 references that are in the map. First posted on arXiv 29 Aug 2024.

Builds on

36 of 43 references.

References · 36

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  8. 2022Video Diffusion Modelscited by 2,280
  9. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  10. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
  11. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  12. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
24 more
  1. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
  2. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  3. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  4. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  5. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 683
  6. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
  7. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
  8. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 573
  9. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
  10. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 462
  11. 2023MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Modelcited by 330
  12. 2023Effective Whole-body Pose Estimation with Two-stages Distillationcited by 307
  13. 2023Composer: Creative and Controllable Image Synthesis with Composable Conditionscited by 303
  14. 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translationcited by 253
  15. 2023TryOnDiffusion: A Tale of Two UNetscited by 141
  16. 2023Advancing Pose-Guided Image Synthesis with Progressive Conditional Diffusion Modelscited by 104
  17. 2023Bridging Vision and Language Encoders: Parameter-Efficient Tuning for Referring Image Segmentationcited by 61
  18. 2024Coarse-to-Fine Latent Diffusion for Pose-Guided Person Image Synthesiscited by 48
  19. 2024MambaTalk: Efficient Holistic Gesture Synthesis with Selective State Space Modelscited by 46
  20. 2023G2L: Semantically Aligned and Uniform Video Grounding via Geodesic and Game Theorycited by 21
  21. 2023Chain of Generation: Multi-Modal Gesture Synthesis via Cascaded Conditional Controlcited by 18
  22. 2024Freetalker: Controllable Speech and Text-Driven Gesture Generation Based on Diffusion Models for Enhanced Speaker Naturalnesscited by 18
  23. 2023JOTR: 3D Joint Contrastive Learning with Transformers for Occluded Human Mesh Recoverycited by 12
  24. 2023Exploiting Auxiliary Caption for Video Groundingcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 8; 2026: 1.

10 papers.