mlsanityData to 14 Oct 2026

Beyond Skeletons: Learning Animation Directly from Driving Videos with Same2X Training Strategy

arXiv 2606.06903 ↗First posted 5 Jun 2026cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Human image animation aims to generate a video from a static reference image, guided by pose information extracted from a driving video. Existing approaches often rely on pose estimators to extract intermediate representations, but such signals are prone to errors under occlusion or complex poses. Building on these observations, we present DirectAnimator, a framework that bypasses pose extraction and directly learns from raw driving videos. We introduce a Driving Cue Triplet consisting of pose, face, and location cues that captures motion, expression, and alignment in a semantically rich yet stable form, and we fuse them through a CueFusion DiT block for reliable control during denoising. To make learning dependable when the driving and reference identities differ, we devise a Same2X training strategy that aligns cross-ID features with those learned from same-ID data, regularizing optimization and accelerating convergence. Extensive experiments demonstrate that DirectAnimator attains state-of-the-art visual quality and identity preservation while remaining robust to occlusions and complex articulation, and it does so with fewer computational resources. Our project page is at https://directanimator.github.io/.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 40 of its 45 references that are in the map. First posted on arXiv 5 Jun 2026.

Builds on

40 of 45 references.

References · 40

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  6. 2022Scalable Diffusion Models with Transformerscited by 6,054
  7. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  8. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  9. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  10. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  11. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  12. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
28 more
  1. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
  2. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,064
  3. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  4. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 703
  5. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 688
  6. 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 652
  7. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 575
  8. 2023MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Modelcited by 330
  9. 2023Effective Whole-body Pose Estimation with Two-stages Distillationcited by 311
  10. 2023Reconstructing Hands in 3D with Transformerscited by 306
  11. 2024Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidancecited by 272
  12. 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformerscited by 225
  13. 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 198
  14. 2022Latent Image Animator: Learning to Animate Images via Latent Space Navigationcited by 175
  15. 2024Identity-Preserving Text-to-Video Generation by Frequency Decompositioncited by 158
  16. 2023DisCo: Disentangled Control for Realistic Human Dance Generationcited by 142
  17. 2024UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animationcited by 102
  18. 2024X-Portrait: Expressive Portrait Animation with Hierarchical Motion Attentioncited by 91
  19. 2024StableAnimator: High-Quality Identity-Preserving Human Image Animationcited by 88
  20. 2024AniTalker: Animate Vivid and Diverse Talking Faces through Identity-Decoupled Facial Motion Encodingcited by 67
  21. 2024Face Adapter for Pre-Trained Diffusion Models with Fine-Grained ID and Attribute Controlcited by 67
  22. 2025No Other Representation Component Is Needed: Diffusion Transformers Can Provide Representation Guidance by Themselvescited by 56
  23. 2025UniAnimate-DiT: Human Image Animation with Large-Scale Video Diffusion Transformercited by 42
  24. 2025HumanDiT: Pose-Guided Diffusion Transformer for Long-form Human Motion Video Generationcited by 39
  25. 2024RealisDance: Equip controllable character animation with realistic handscited by 37
  26. 2025DreamActor-M1: Holistic, Expressive and Robust Human Image Animation with Hybrid Guidancecited by 36
  27. 2025X-Dyna: Expressive Dynamic Human Image Animationcited by 18
  28. 2025DynamiCtrl: Rethinking the Basic Structure and the Role of Text for High-quality Human Image Animationcited by 7

Built on by

Not cited yet.