mlsanityData to 14 Oct 2026

Animating the Uncaptured: Humanoid Mesh Animation with Video Diffusion Models

arXiv 2503.15996 ↗First posted 20 Mar 2025cs.GR cs.CVcited by 3

On the map: Diffusion models → Video generation

Abstract (arXiv)

Animation of humanoid characters is essential in various graphics applications, but requires significant time and cost to create realistic animations. We propose an approach to synthesize 4D animated sequences of input static 3D humanoid meshes, leveraging strong generalized motion priors from generative video models -- as such video models contain powerful motion information covering a wide variety of human motions. From an input static 3D humanoid mesh and a text prompt describing the desired animation, we synthesize a corresponding video conditioned on a rendered image of the 3D mesh. We then employ an underlying SMPL representation to animate the corresponding 3D mesh according to the video-generated motion, based on our motion optimization. This enables a cost-effective and accessible solution to enable the synthesis of diverse and realistic 4D animations.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 34 of its 55 references that are in the map. First posted on arXiv 20 Mar 2025.

Builds on

34 of 55 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
  2. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
  3. 2023Human Motion Diffusion as a Generative Prior2 Mar
  4. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model31 Aug

Other references · 30

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  3. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  4. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
  5. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,162
  6. 2017End-to-end Recovery of Human Shape and Posecited by 885
  7. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 784
  8. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 695
  9. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 627
  10. 2019MediaPipe: A Framework for Building Perception Pipelinescited by 583
  11. 2019NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understandingcited by 533
  12. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 441
18 more
  1. 2023Humans in 4D: Reconstructing and Tracking Humans with Transformerscited by 350
  2. 2016The KIT Motion-Language Datasetcited by 338
  3. 2023A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondencecited by 301
  4. 2024Sapiens: Foundation for Human Vision Modelscited by 229
  5. 2023Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondencecited by 214
  6. 2023WHAM: Reconstructing World-grounded Humans with Accurate 3D Motioncited by 206
  7. 2020STAR: Sparse Trained Articulated Human Body Regressorcited by 194
  8. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 161
  9. 2024SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistencycited by 144
  10. 2015DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimationcited by 117
  11. 2025VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Modelscited by 113
  12. 20244Diffusion: Multi-view Video Diffusion Model for 4D Generationcited by 104
  13. 2023Diffusion 3D Features (Diff3F): Decorating Untextured Shapes with Distilled Semantic Featurescited by 55
  14. 2023GenZI: Zero-Shot 3D Human-Scene Interaction Generationcited by 40
  15. 2024ZeroHSI: Zero-Shot 4D Human-Scene Interaction by Video Generationcited by 25
  16. 2020Polarization Human Shape and Pose Datasetcited by 10
  17. 2024SMPLX-Lite: A Realistic and Drivable Avatar Benchmark with Rich Geometry and Texture Annotationscited by 7
  18. 2024Stereo-Talker: Audio-driven 3D Human Synthesis with Prior-Guided Mixture-of-Expertscited by 4

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025Puppeteer: Rig and Animate Your 3D Modelscited by 43
  2. 2025The Quest for Generalizable Motion Generation: Data, Model, and Evaluationcited by 22
  3. 2025AnimaX: Animating the Inanimate in 3D with Joint Video-Pose Diffusion Modelscited by 15

Similar, not cited

Close in wording, no citation either way

  1. 2025AnimaMimic: Imitating 3D Animation from Video Priors
  2. 2026Ani3DHuman: Photorealistic 3D Human Animation with Self-guided Stochastic Sampling
  3. 2026Sketch2Motion: Text-driven 2D Sketch to 3D Animation via Diffusion-guided Skeleton Optimization
  4. 2026ActionMesh: Animated 3D Mesh Generation with Temporal 3D Diffusion
  5. 2024MotionDreamer: Exploring Semantic Video Diffusion features for Zero-Shot 3D Mesh Animation
  6. 2024UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animation
  7. 2025AnimateAnyMesh: A Feed-Forward 4D Foundation Model for Text-Driven Universal Mesh Animation
  8. 2024Animate3D: Animating Any 3D Model with Multi-view Video Diffusion