# Puppeteer: Rig and Animate Your 3D Models

arXiv: https://arxiv.org/abs/2508.10898 · HTML: https://mlsanity.com/paper/2508.10898/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 79 of its 105 references that are in the map.
- First posted on arXiv 14 Aug 2025.

## Builds on (most cited first, 30 of 79)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model](https://mlsanity.com/paper/2208.15001/) (2022)
- [Objaverse-XL: A Universe of 10M+ 3D Objects](https://mlsanity.com/paper/2307.05663/) (2023)
- [MotionGPT: Human Motion as a Foreign Language](https://mlsanity.com/paper/2306.14795/) (2023)
- [MotionCLIP: Exposing Human Motion Generation to CLIP Space](https://mlsanity.com/paper/2203.08063/) (2022)
- [3D Menagerie: Modeling the 3D shape and pose of animals](https://mlsanity.com/paper/1611.07700/) (2016)
- [MeshGPT: Generating Triangle Meshes with Decoder-Only Transformers](https://mlsanity.com/paper/2311.15475/) (2023)
- [Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representation](https://mlsanity.com/paper/2306.17115/) (2023)
- [Video Depth Anything: Consistent Depth Estimation for Super-Long Videos](https://mlsanity.com/paper/2501.12375/) (2025)
- [BANMo: Building Animatable 3D Neural Models from Many Casual Videos](https://mlsanity.com/paper/2112.12761/) (2021)
- [DreamGaussian4D: Generative 4D Gaussian Splatting](https://mlsanity.com/paper/2312.17142/) (2023)
- [4D-fy: Text-to-4D Generation Using Hybrid Score Distillation Sampling](https://mlsanity.com/paper/2311.17984/) (2023)
- [Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Models](https://mlsanity.com/paper/2312.13763/) (2023)
- [MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformers](https://mlsanity.com/paper/2406.10163/) (2024)
- [L4GM: Large 4D Gaussian Reconstruction Model](https://mlsanity.com/paper/2406.10324/) (2024)
- [SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistency](https://mlsanity.com/paper/2407.17470/) (2024)
- [CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videos](https://mlsanity.com/paper/2410.11831/) (2024)
- [Randomized Autoregressive Visual Generation](https://mlsanity.com/paper/2411.00776/) (2024)
- [Consistent4D: Consistent 360deg Dynamic Object Generation from Monocular Video](https://mlsanity.com/paper/2311.02848/) (2023)
- [4DGen: Grounded 4D Content Generation with Spatial-temporal Consistency](https://mlsanity.com/paper/2312.17225/) (2023)
- [Animate124: Animating One Image to 4D Dynamic Scene](https://mlsanity.com/paper/2311.14603/) (2023)
- [MagicPony: Learning Articulated 3D Animals in the Wild](https://mlsanity.com/paper/2211.12497/) (2022)
- [4Diffusion: Multi-view Video Diffusion Model for 4D Generation](https://mlsanity.com/paper/2405.20674/) (2024)
- [MeshAnything V2: Artist-Created Mesh Generation With Adjacent Mesh Tokenization](https://mlsanity.com/paper/2408.02555/) (2024)
- [STAG4D: Spatial-Temporal Anchored Generative 4D Gaussians](https://mlsanity.com/paper/2403.14939/) (2024)
- [GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creation](https://mlsanity.com/paper/2403.12365/) (2024)

## Built on by (most cited first, 30 of 43)

- [Motion 3-to-4: 3D Motion Reconstruction for 4D Synthesis](https://mlsanity.com/paper/2601.14253/) (2026)
- [SS4D: Native 4D Generative Model via Structured Spacetime Latents](https://mlsanity.com/paper/2512.14284/) (2025)
- [AnimaMimic: Imitating 3D Animation from Video Priors](https://mlsanity.com/paper/2512.14133/) (2025)
- [One Video, One World: Turning Monocular Video into Physical 4D Scenes](https://mlsanity.com/paper/2606.31388/) (2026)
- [Engine-Native Editable 3D World Reconstruction with Objects and Lighting](https://mlsanity.com/paper/2607.20889/) (2026)
- [Choreographing a World of Dynamic Objects](https://mlsanity.com/paper/2601.04194/) (2026)
- [Skin Tokens: A Learned Compact Representation for Unified Autoregressive Rigging](https://mlsanity.com/paper/2602.04805/) (2026)
- [LottieGPT: Tokenizing Vector Animation for Autoregressive Generation](https://mlsanity.com/paper/2604.11792/) (2026)
- [Full-4D: Generating Full-Scope 4D Scenes from a Single-View Video](https://mlsanity.com/paper/2605.25500/) (2026)
- [Advances in 4D Representation: Geometry, Motion, and Interaction](https://mlsanity.com/paper/2510.19255/) (2025)
- [BiMotion: B-spline Motion for Text-guided Dynamic 3D Character Generation](https://mlsanity.com/paper/2602.18873/) (2026)
- [Seed3D 2.0: Advancing High-Fidelity Simulation-Ready 3D Content Generation](https://mlsanity.com/paper/2605.13862/) (2026)
- [AniGen: Unified S3 Fields for Animatable 3D Asset Generation](https://mlsanity.com/paper/2604.08746/) (2026)
- [ViPS: Video-informed Pose Spaces for Auto-Rigged Meshes](https://mlsanity.com/paper/2604.17623/) (2026)
- [Stroke3D: Lifting 2D strokes into rigged 3D model via latent diffusion models](https://mlsanity.com/paper/2602.09713/) (2026)
- [MimiCAT: Mimic with Correspondence-Aware Cascade-Transformer for Category-Free 3D Pose Transfer](https://mlsanity.com/paper/2511.18370/) (2025)
- [MorphGS: Morphology-Adaptive Articulated 3D Motion Transfer from Videos](https://mlsanity.com/paper/2601.02716/) (2026)
- [SPRig: Self-Supervised Pose-Invariant Rigging from Mesh Sequences](https://mlsanity.com/paper/2602.12740/) (2026)
- [SK-Adapter: Skeleton-Based Structural Control for Native 3D Generation](https://mlsanity.com/paper/2603.14152/) (2026)
- [DancingBox: A Lightweight MoCap System for Character Animation from Physical Proxies](https://mlsanity.com/paper/2603.17704/) (2026)
- [Sketch2Motion: Text-driven 2D Sketch to 3D Animation via Diffusion-guided Skeleton Optimization](https://mlsanity.com/paper/2605.28394/) (2026)
- [Follow Your Track: Precise Skeleton Animation Controlled by 3D Trajectories](https://mlsanity.com/paper/2606.25344/) (2026)
- [SAMoR: Motion Modelling for Articulated Objects of Any Skeleton and Topology](https://mlsanity.com/paper/2607.02148/) (2026)
- [ViP-Rig: Visual-Prompted Controllable Rigging](https://mlsanity.com/paper/2607.27982/) (2026)
- [Kirin: Animal Motion Generation from In-the-Wild Video](https://mlsanity.com/paper/2609.01823/) (2026)
- [UniMate: One Unified Model to Animate Diverse Skeletons](https://mlsanity.com/paper/2609.05415/) (2026)
- [3DProxyImg: Controllable 3D-Aware Animation Synthesis from Single Image via 2D-3D Aligned Proxy Embedding](https://mlsanity.com/paper/2512.15126/) (2025)
- [Make-It-Poseable: Feed-forward Latent Posing Model for 3D Characters](https://mlsanity.com/paper/2512.16767/) (2025)
- [Muses: Designing, Composing, Generating Nonexistent Fantasy 3D Creatures without Training](https://mlsanity.com/paper/2601.03256/) (2026)
- [PhysSkin: Real-Time and Generalizable Physics-Based Animation via Self-Supervised Neural Skinning](https://mlsanity.com/paper/2603.23194/) (2026)

## Abstract (arXiv)

Modern interactive applications increasingly demand dynamic 3D content, yet the transformation of static 3D models into animated assets constitutes a significant bottleneck in content creation pipelines. While recent advances in generative AI have revolutionized static 3D model creation, rigging and animation continue to depend heavily on expert intervention. We present Puppeteer, a comprehensive framework that addresses both automatic rigging and animation for diverse 3D objects. Our system first predicts plausible skeletal structures via an auto-regressive transformer that introduces a joint-based tokenization strategy for compact representation and a hierarchical ordering methodology with stochastic perturbation that enhances bidirectional learning capabilities. It then infers skinning weights via an attention-based architecture incorporating topology-aware joint attention that explicitly encodes inter-joint relationships based on skeletal graph distances. Finally, we complement these rigging advances with a differentiable optimization-based animation pipeline that generates stable, high-fidelity animations while being computationally more efficient than existing approaches. Extensive evaluations across multiple benchmarks demonstrate that our method significantly outperforms state-of-the-art techniques in both skeletal prediction accuracy and skinning quality. The system robustly processes diverse 3D content, ranging from professionally designed game assets to AI-generated shapes, producing temporally coherent animations that eliminate the jittering issues common in existing methods.