Beyond Skeletons: Learning Animation Directly from Driving Videos with Same2X Training Strategy
Not on the map. Why?
Abstract (arXiv)
Human image animation aims to generate a video from a static reference image, guided by pose information extracted from a driving video. Existing approaches often rely on pose estimators to extract intermediate representations, but such signals are prone to errors under occlusion or complex poses. Building on these observations, we present DirectAnimator, a framework that bypasses pose extraction and directly learns from raw driving videos. We introduce a Driving Cue Triplet consisting of pose, face, and location cues that captures motion, expression, and alignment in a semantically rich yet stable form, and we fuse them through a CueFusion DiT block for reliable control during denoising. To make learning dependable when the driving and reference identities differ, we devise a Same2X training strategy that aligns cross-ID features with those learned from same-ID data, regularizing optimization and accelerating convergence. Extensive experiments demonstrate that DirectAnimator attains state-of-the-art visual quality and identity preservation while remaining robust to occlusions and complex articulation, and it does so with fewer computational resources. Our project page is at https://directanimator.github.io/.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 40 of its 45 references that are in the map. First posted on arXiv 5 Jun 2026.
Builds on
40 of 45 references.
References · 40
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
- 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
28 more
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
- 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,064
- 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
- 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 703
- 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 688
- 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 652
- 2018DensePose: Dense Human Pose Estimation In The Wildcited by 575
- 2023MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Modelcited by 330
- 2023Effective Whole-body Pose Estimation with Two-stages Distillationcited by 311
- 2023Reconstructing Hands in 3D with Transformerscited by 306
- 2024Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidancecited by 272
- 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformerscited by 225
- 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 198
- 2022Latent Image Animator: Learning to Animate Images via Latent Space Navigationcited by 175
- 2024Identity-Preserving Text-to-Video Generation by Frequency Decompositioncited by 158
- 2023DisCo: Disentangled Control for Realistic Human Dance Generationcited by 142
- 2024UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animationcited by 102
- 2024X-Portrait: Expressive Portrait Animation with Hierarchical Motion Attentioncited by 91
- 2024StableAnimator: High-Quality Identity-Preserving Human Image Animationcited by 88
- 2024AniTalker: Animate Vivid and Diverse Talking Faces through Identity-Decoupled Facial Motion Encodingcited by 67
- 2024Face Adapter for Pre-Trained Diffusion Models with Fine-Grained ID and Attribute Controlcited by 67
- 2025No Other Representation Component Is Needed: Diffusion Transformers Can Provide Representation Guidance by Themselvescited by 56
- 2025UniAnimate-DiT: Human Image Animation with Large-Scale Video Diffusion Transformercited by 42
- 2025HumanDiT: Pose-Guided Diffusion Transformer for Long-form Human Motion Video Generationcited by 39
- 2024RealisDance: Equip controllable character animation with realistic handscited by 37
- 2025DreamActor-M1: Holistic, Expressive and Robust Human Image Animation with Hybrid Guidancecited by 36
- 2025X-Dyna: Expressive Dynamic Human Image Animationcited by 18
- 2025DynamiCtrl: Rethinking the Basic Structure and the Role of Text for High-quality Human Image Animationcited by 7
Built on by
Not cited yet.