T2Bs: Text-to-Character Blendshapes via Video Generation
On the map: Diffusion models → 3D, point cloud and scene generation
Abstract (arXiv)
We present T2Bs, a framework for generating high-quality, animatable character head morphable models from text by combining static text-to-3D generation with video diffusion. Text-to-3D models produce detailed static geometry but lack motion synthesis, while video diffusion models generate motion with temporal and multi-view geometric inconsistencies. T2Bs bridges this gap by leveraging deformable 3D Gaussian splatting to align static 3D assets with video outputs. By constraining motion with static geometry and employing a view-dependent deformation MLP, T2Bs (i) outperforms existing 4D generation methods in accuracy and expressiveness while reducing video artifacts and view inconsistencies, and (ii) reconstructs smooth, coherent, fully registered 3D geometries designed to scale for building morphable models with diverse, realistic facial motions. This enables synthesizing expressive, animatable character heads that surpass current 4D generation techniques.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 87 of its 112 references that are in the map. First posted on arXiv 12 Sep 2025.
Builds on
87 of 112 references.
Closest ancestors · 32
Same direction, newest first
- 2024CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Models16 Dec
- 20244Real-Video: Learning Generalizable Photo-Realistic 4D Video Diffusion5 Dec
- 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models27 Nov
- 2024SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistency24 Jul
- 2024Controlling Space and Time with Diffusion Models10 Jul
- 2024RodinHD: High-Fidelity 3D Avatar Generation with Diffusion Models9 Jul
- 20244Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Models11 Jun
- 20244Diffusion: Multi-view Video Diffusion Model for 4D Generation31 May
- 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Models16 May
- 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion18 Mar
- 20234DGen: Grounded 4D Content Generation with Spatial-temporal Consistency28 Dec
- 2023Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Models21 Dec
20 more
- 2023PrimDiffusion: Volumetric Primitives Diffusion for 3D Human Generation7 Dec
- 2023Animate124: Animating One Image to 4D Dynamic Scene24 Nov
- 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model10 Nov
- 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model23 Oct
- 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models12 Oct
- 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
- 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors30 Jun
- 2023HeadSculpt: Crafting 3D Head Avatars with Text5 Jun
- 2023AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatars1 Jun
- 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
- 2023TextMesh: Generation of Realistic 3D Meshes From Text Prompts24 Apr
- 2023Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstruction13 Apr
- 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior24 Mar
- 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
- 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
- 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
- 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion12 Dec
- 2022DiffRF: Rendering-Guided 3D Radiance Field Diffusion2 Dec
- 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
- 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
Other references · 55
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2023GPT-4 Technical Reportcited by 18,396
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
- 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
- 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
- 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
38 more
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
- 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,157
- 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 843
- 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 774
- 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 770
- 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
- 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 535
- 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 460
- 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 460
- 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 433
- 2023Text-to-3D using Gaussian Splattingcited by 299
- 2023LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matchingcited by 290
- 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 288
- 20193D Morphable Face Models -- Past, Present and Futurecited by 270
- 2023GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussianscited by 260
- 2023Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformerscited by 258
- 2024Shape of Motion: 4D Reconstruction from a Single Videocited by 250
- 2023Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representationcited by 228
- 2023Text-To-4D Dynamic Scene Generationcited by 205
- 2021BANMo: Building Animatable 3D Neural Models from Many Casual Videoscited by 197
- 2023DreamGaussian4D: Generative 4D Gaussian Splattingcited by 194
- 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 185
- 2024MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffoldscited by 179
- 2024SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splattingcited by 172
- 2024Direct-a-Video: Customized Video Generation with User-Directed Camera Movement and Object Motioncited by 167
- 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
- 2024GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generationcited by 153
- 2024L4GM: Large 4D Gaussian Reconstruction Modelcited by 144
- 2022Neural Wavelet-domain Diffusion for 3D Shape Generationcited by 132
- 2023Consistent4D: Consistent 360deg Dynamic Object Generation from Monocular Videocited by 125
- 2021LASR: Learning Articulated Shape Reconstruction from a Monocular Videocited by 116
- 2023FlashAvatar: High-fidelity Head Avatar with Efficient Gaussian Embeddingcited by 108
- 2024Snap Video: Scaled Spatiotemporal Transformers for Text-to-Video Synthesiscited by 105
- 2023GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learningcited by 63
- 2024SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfercited by 61
- 2024DreamMesh4D: Video-to-4D Generation with Sparse-Controlled Gaussian-Mesh Hybrid Representationcited by 58
- 2023Flow supervision for Deformable NeRFcited by 42
- 2024Vidu4D: Single Generated Video to High-Fidelity 4D Reconstruction with Dynamic Gaussian Surfelscited by 38
50 most cited of 55 All 55 (JSON)
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2023Articulated 3D Head Avatar Generation using Text-to-Image Diffusion Models
- 2025Text-based Animatable 3D Avatars with Morphable Model Alignment
- 2025StyleMM: Stylized 3D Morphable Face Model via Text-Driven Aligned Image Translation
- 2026Rotate Your Character: Revisiting Video Diffusion Models for High-Quality 3D Character Generation
- 2023Text-Guided 3D Face Synthesis -- From Generation to Editing
- 2023DreamFace: Progressive Generation of Animatable 3D Faces under Text Guidance
- 2024Morphable Diffusion: 3D-Consistent Diffusion for Single-image Avatar Creation
- 2024CT4D: Consistent Text-to-4D Generation with Animatable Meshes