# MagicPose4D: Crafting Articulated Models with Appearance and Motion Control

arXiv: https://arxiv.org/abs/2405.14017 · HTML: https://mlsanity.com/paper/2405.14017/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 52 references that are in the map.
- First posted on arXiv 22 May 2024.

## Builds on (most cited first, 30 of 38)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation](https://mlsanity.com/paper/2309.16653/) (2023)
- [3D Menagerie: Modeling the 3D shape and pose of animals](https://mlsanity.com/paper/1611.07700/) (2016)
- [EMOCA: Emotion Driven Monocular Face Capture and Animation](https://mlsanity.com/paper/2204.11312/) (2022)
- [Text-To-4D Dynamic Scene Generation](https://mlsanity.com/paper/2301.11280/) (2023)
- [BANMo: Building Animatable 3D Neural Models from Many Casual Videos](https://mlsanity.com/paper/2112.12761/) (2021)
- [DreamGaussian4D: Generative 4D Gaussian Splatting](https://mlsanity.com/paper/2312.17142/) (2023)
- [4D-fy: Text-to-4D Generation Using Hybrid Score Distillation Sampling](https://mlsanity.com/paper/2311.17984/) (2023)
- [Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Models](https://mlsanity.com/paper/2312.13763/) (2023)
- [Consistent4D: Consistent 360deg Dynamic Object Generation from Monocular Video](https://mlsanity.com/paper/2311.02848/) (2023)
- [Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfer](https://mlsanity.com/paper/2311.17009/) (2023)
- [LASR: Learning Articulated Shape Reconstruction from a Monocular Video](https://mlsanity.com/paper/2105.02976/) (2021)
- [Animate124: Animating One Image to 4D Dynamic Scene](https://mlsanity.com/paper/2311.14603/) (2023)
- [MagicPony: Learning Articulated 3D Animals in the Wild](https://mlsanity.com/paper/2211.12497/) (2022)
- [STAG4D: Spatial-Temporal Anchored Generative 4D Gaussians](https://mlsanity.com/paper/2403.14939/) (2024)
- [GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creation](https://mlsanity.com/paper/2403.12365/) (2024)
- [NPMs: Neural Parametric Models for 3D Deformable Shapes](https://mlsanity.com/paper/2104.00702/) (2021)
- [Creatures great and SMAL: Recovering the shape and motion of animals from video](https://mlsanity.com/paper/1811.05804/) (2018)
- [A Unified Approach for Text- and Image-guided 4D Scene Generation](https://mlsanity.com/paper/2311.16854/) (2023)
- [TC4D: Trajectory-Conditioned Text-to-4D Generation](https://mlsanity.com/paper/2403.17920/) (2024)
- [Learning Neural Parametric Head Models](https://mlsanity.com/paper/2212.02761/) (2022)
- [3D Pose Transfer with Correspondence Learning and Mesh Refinement](https://mlsanity.com/paper/2109.15025/) (2021)
- [Skeleton-free Pose Transfer for Stylized 3D Characters](https://mlsanity.com/paper/2208.00790/) (2022)
- [Unsupervised 3D Pose Transfer with Cross Consistency and Dual Reconstruction](https://mlsanity.com/paper/2211.10278/) (2022)
- [CSL: Class-Agnostic Structure-Constrained Learning for Segmentation Including the Unseen](https://mlsanity.com/paper/2312.05538/) (2023)
- [OOSTraj: Out-of-Sight Trajectory Prediction With Vision-Positioning Denoising](https://mlsanity.com/paper/2404.02227/) (2024)

## Built on by (most cited first, 18 of 18)

- [VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control](https://mlsanity.com/paper/2407.12781/) (2024)
- [AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformers](https://mlsanity.com/paper/2411.18673/) (2024)
- [MagicArticulate: Make Your 3D Models Articulation-Ready](https://mlsanity.com/paper/2502.12135/) (2025)
- [Puppeteer: Rig and Animate Your 3D Models](https://mlsanity.com/paper/2508.10898/) (2025)
- [Motion Anything: Any to Motion Generation](https://mlsanity.com/paper/2503.06955/) (2025)
- [Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation](https://mlsanity.com/paper/2509.19296/) (2025)
- [Drive Any Mesh: 4D Latent Diffusion for Mesh Deformation from Video](https://mlsanity.com/paper/2506.07489/) (2025)
- [PhysRig: Differentiable Physics-Based Skinning and Rigging Framework for Realistic Articulated Object Modeling](https://mlsanity.com/paper/2506.20936/) (2025)
- [Auto-Connect: Connectivity-Preserving RigFormer with Direct Preference Optimization](https://mlsanity.com/paper/2506.11430/) (2025)
- [One Video, One World: Turning Monocular Video into Physical 4D Scenes](https://mlsanity.com/paper/2606.31388/) (2026)
- [ARMO: Autoregressive Rigging for Multi-Category Objects](https://mlsanity.com/paper/2503.20663/) (2025)
- [Divide and Fuse: Body Part Mesh Recovery from Partially Visible Human Images](https://mlsanity.com/paper/2407.09694/) (2024)
- [UniMC: Taming Diffusion Transformer for Unified Keypoint-Guided Multi-Class Image Generation](https://mlsanity.com/paper/2507.02713/) (2025)
- [Part2GS: Part-aware Modeling of Articulated Objects using 3D Gaussian Splatting](https://mlsanity.com/paper/2506.17212/) (2025)
- [MimiCAT: Mimic with Correspondence-Aware Cascade-Transformer for Category-Free 3D Pose Transfer](https://mlsanity.com/paper/2511.18370/) (2025)
- [MorphGS: Morphology-Adaptive Articulated 3D Motion Transfer from Videos](https://mlsanity.com/paper/2601.02716/) (2026)
- [SPRig: Self-Supervised Pose-Invariant Rigging from Mesh Sequences](https://mlsanity.com/paper/2602.12740/) (2026)
- [Motion Marionette: Rethinking Rigid Motion Transfer via Prior Guidance](https://mlsanity.com/paper/2511.19909/) (2025)

## Abstract (arXiv)

With the success of 2D and 3D visual generative models, there is growing interest in generating 4D content. Existing methods primarily rely on text prompts to produce 4D content, but they often fall short of accurately defining complex or rare motions. To address this limitation, we propose MagicPose4D, a novel framework for refined control over both appearance and motion in 4D generation. Unlike current 4D generation methods, MagicPose4D accepts monocular videos or mesh sequences as motion prompts, enabling precise and customizable motion control. MagicPose4D comprises two key modules: (i) Dual-Phase 4D Reconstruction Module, which operates in two phases. The first phase focuses on capturing the model's shape using accurate 2D supervision and less accurate but geometrically informative 3D pseudo-supervision without imposing skeleton constraints. The second phase extracts the 3D motion (skeleton poses) using more accurate pseudo-3D supervision, obtained in the first phase and introduces kinematic chain-based skeleton constraints to ensure physical plausibility. Additionally, we propose a Global-local Chamfer loss that aligns the overall distribution of predicted mesh vertices with the supervision while maintaining part-level alignment without extra annotations. (ii) Cross-category Motion Transfer Module, which leverages the extracted motion from the 4D reconstruction module and uses a kinematic-chain-based skeleton to achieve cross-category motion transfer. It ensures smooth transitions between frames through dynamic rigidity, facilitating robust generalization without additional training. Through extensive experiments, we demonstrate that MagicPose4D significantly improves the accuracy and consistency of 4D content generation, outperforming existing methods in various benchmarks.