R-DMesh: Video-Guided 3D Animation via Rectified Dynamic Mesh Flow
On the map: Diffusion models → Video generation
Abstract (arXiv)
Video-guided 3D animation holds immense potential for content creation, offering intuitive and precise control over dynamic assets. However, practical deployment faces a critical yet frequently overlooked hurdle: the pose misalignment dilemma. In real-world scenarios, the initial pose of a user-provided static mesh rarely aligns with the starting frame of a reference video. Naively forcing a mesh to follow a mismatched trajectory inevitably leads to severe geometric distortion or animation failure. To address this, we present Rectified Dynamic Mesh (R-DMesh), a unified framework designed to generate high-fidelity 4D meshes that are ``rectified'' to align with video context. Unlike standard motion transfer approaches, our method introduces a novel VAE that explicitly disentangles the input into a conditional base mesh, relative motion trajectories, and a crucial rectification jump offset. This offset is learned to automatically transform the arbitrary pose of the input mesh to match the video's initial state before animation begins. We process these components via a Triflow Attention mechanism, which leverages vertex-wise geometric features to modulate the three orthogonal flows, ensuring physical consistency and local rigidity during the rectification and animation process. For generation, we employ a Rectified Flow-based Diffusion Transformer conditioned on pre-trained video latents, effectively transferring rich spatio-temporal priors to the 3D domain. To support this task, we construct Video-RDMesh, a large-scale dataset of over 500k dynamic mesh sequences specifically curated to simulate pose misalignment. Extensive experiments demonstrate that R-DMesh not only solves the alignment problem but also enables robust downstream applications, including pose retargeting and holistic 4D generation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 35 of its 46 references that are in the map. First posted on arXiv 13 May 2026.
Builds on
35 of 46 references.
Closest ancestors · 2
Same direction, newest first
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
- 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model31 Aug
Other references · 33
- 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
- 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
- 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,465
- 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,157
- 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
- 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 774
- 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
21 more
- 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 588
- 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 521
- 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Modelscited by 458
- 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 455
- 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 400
- 2023Text-To-4D Dynamic Scene Generationcited by 205
- 2024L4GM: Large 4D Gaussian Reconstruction Modelcited by 144
- 20244Diffusion: Multi-view Video Diffusion Model for 4D Generationcited by 104
- 2024Diffusion4D: Fast Spatial-temporal Consistent 4D Generation via Video Diffusion Modelscited by 100
- 2024SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfercited by 61
- 2024Animate3D: Animating Any 3D Model with Multi-view Video Diffusioncited by 56
- 2025Puppeteer: Rig and Animate Your 3D Modelscited by 43
- 2025AnyTop: Character Animation Diffusion with Any Topologycited by 41
- 2025V2M4: 4D Mesh Animation Reconstruction from a Single Monocular Videocited by 33
- 2025AnimateAnyMesh: A Feed-Forward 4D Foundation Model for Text-Driven Universal Mesh Animationcited by 32
- 2022Neural Capture of Animatable 3D Human from Monocular Videocited by 22
- 2024CT4D: Consistent Text-to-4D Generation with Animatable Meshescited by 14
- 2025Drive Any Mesh: 4D Latent Diffusion for Mesh Deformation from Videocited by 12
- 2026Mesh4D: 4D Mesh Reconstruction and Tracking from Monocular Videocited by 12
- 2025MoCapAnything: Unified 3D Motion Capture for Arbitrary Skeletons from Monocular Videoscited by 10
- 2026AnimateAnyMesh++: A Flexible Feed-Forward Framework for High-Fidelity Text-Driven Mesh Animationcited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026ActionMesh: Animated 3D Mesh Generation with Temporal 3D Diffusion
- 2023Generative Rendering: Controllable 4D-Guided Video Generation with 2D Diffusion Models
- 2026Feed-forward Motion In-betweening for Any 4D
- 2025Shape-for-Motion: Precise and Consistent Video Editing with 3D Proxy
- 2026ViPS: Video-informed Pose Spaces for Auto-Rigged Meshes
- 20264Director: Controlling Video World Models with Rigid 3D Geometry
- 2024MotionDreamer: Exploring Semantic Video Diffusion features for Zero-Shot 3D Mesh Animation
- 2026Towards 3D-Aware Video Diffusion Models: Render-Free Human Motion Control with Mesh Tokenization