# CharacterGen: Efficient 3D Character Generation from Single Images with Multi-View Pose Canonicalization

arXiv: https://arxiv.org/abs/2402.17214 · HTML: https://mlsanity.com/paper/2402.17214/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 67 of its 83 references that are in the map.
- First posted on arXiv 27 Feb 2024.

## Builds on (most cited first, 30 of 67)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [MVDream: Multi-view Diffusion for 3D Generation](https://mlsanity.com/paper/2308.16512/) (2023)
- [LRM: Large Reconstruction Model for Single Image to 3D](https://mlsanity.com/paper/2311.04400/) (2023)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [Point-E: A System for Generating 3D Point Clouds from Complex Prompts](https://mlsanity.com/paper/2212.08751/) (2022)
- [Wonder3D: Single Image to 3D using Cross-Domain Diffusion](https://mlsanity.com/paper/2310.15008/) (2023)
- [Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation](https://mlsanity.com/paper/2311.17117/) (2023)
- [Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation](https://mlsanity.com/paper/2303.13873/) (2023)
- [SyncDreamer: Generating Multiview-consistent Images from a Single-view Image](https://mlsanity.com/paper/2309.03453/) (2023)
- [Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation](https://mlsanity.com/paper/2212.00774/) (2022)
- [One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization](https://mlsanity.com/paper/2306.16928/) (2023)
- [LION: Latent Point Diffusion Models for 3D Shape Generation](https://mlsanity.com/paper/2210.06978/) (2022)
- [PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization](https://mlsanity.com/paper/2004.00452/) (2020)
- [Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model](https://mlsanity.com/paper/2310.15110/) (2023)
- [Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures](https://mlsanity.com/paper/2211.07600/) (2022)
- [Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis](https://mlsanity.com/paper/2111.04276/) (2021)
- [3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models](https://mlsanity.com/paper/2301.11445/) (2023)
- [Modular Primitives for High-Performance Differentiable Rendering](https://mlsanity.com/paper/2011.03277/) (2020)
- [Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors](https://mlsanity.com/paper/2306.17843/) (2023)

## Built on by (most cited first, 28 of 28)

- [LHM: Large Animatable Human Reconstruction Model from a Single Image in Seconds](https://mlsanity.com/paper/2503.10625/) (2025)
- [IDOL: Instant Photorealistic 3D Human Creation from a Single Image](https://mlsanity.com/paper/2412.14963/) (2024)
- [HumanSplat: Generalizable Single-Image Human Gaussian Splatting with Structure Priors](https://mlsanity.com/paper/2406.12459/) (2024)
- [AniGS: Animatable Gaussian Avatar from a Single Image with Inconsistent Gaussian Reconstruction](https://mlsanity.com/paper/2412.02684/) (2024)
- [DRiVE: Diffusion-based Rigging Empowers Generation of Versatile and Expressive Characters](https://mlsanity.com/paper/2411.17423/) (2024)
- [HumanDreamer-X: Photorealistic Single-image Human Avatars Reconstruction via Gaussian Restoration](https://mlsanity.com/paper/2504.03536/) (2025)
- [SIGMAN:Scaling 3D Human Gaussian Generation with Millions of Assets](https://mlsanity.com/paper/2504.06982/) (2025)
- [ATLAS: Decoupling Skeletal and Shape Parameters for Expressive Parametric Human Modeling](https://mlsanity.com/paper/2508.15767/) (2025)
- [UP2You: Fast Reconstruction of Yourself from Unconstrained Photo Collections](https://mlsanity.com/paper/2509.24817/) (2025)
- [CharacterShot: Controllable and Consistent 4D Character Animation](https://mlsanity.com/paper/2508.07409/) (2025)
- [MHR: Momentum Human Rig](https://mlsanity.com/paper/2511.15586/) (2025)
- [AdaHuman: Animatable Detailed 3D Human Generation with Compositional Multiview Diffusion](https://mlsanity.com/paper/2505.24877/) (2025)
- [StdGEN: Semantic-Decomposed 3D Character Generation from Single Images](https://mlsanity.com/paper/2411.05738/) (2024)
- [Dream, Lift, Animate: From Single Images to Animatable Gaussian Avatars](https://mlsanity.com/paper/2507.15979/) (2025)
- [PoseMaster: A Unified 3D Native Framework for Stylized Pose Generation](https://mlsanity.com/paper/2506.21076/) (2025)
- [Occlusion-robust Stylization for Drawing-based 3D Animation](https://mlsanity.com/paper/2508.00398/) (2025)
- [See-through: Single-image Layer Decomposition for Anime Characters](https://mlsanity.com/paper/2602.03749/) (2026)
- [PARTE: Part-Guided Texturing for 3D Human Reconstruction from a Single Image](https://mlsanity.com/paper/2507.17332/) (2025)
- [AHOY! Animatable Humans under Occlusion from YouTube Videos with Gaussian Splatting and Video Diffusion Priors](https://mlsanity.com/paper/2603.17975/) (2026)
- [DreamCharacter-1: From 3D Generative Foundation Models to Product-Ready Character Generation](https://mlsanity.com/paper/2607.07817/) (2026)
- [Rotate Your Character: Revisiting Video Diffusion Models for High-Quality 3D Character Generation](https://mlsanity.com/paper/2601.05722/) (2026)
- [StdGEN++: A Comprehensive System for Semantic-Decomposed 3D Character Generation](https://mlsanity.com/paper/2601.07660/) (2026)
- [TopoCap: Learning Topology-Agnostic Motion Priors for Monocular Video-to-Animation](https://mlsanity.com/paper/2606.12153/) (2026)
- [SmartAvatar: Text- and Image-Guided Human Avatar Generation with VLM AI Agents](https://mlsanity.com/paper/2506.04606/) (2025)
- [Blur2Sharp: Human Novel Pose and View Synthesis with Generative Prior Refinement](https://mlsanity.com/paper/2512.08215/) (2025)
- [Dehallu3D: Hallucination-Mitigated 3D Generation from Single Image via Cyclic View Consistency Refinement](https://mlsanity.com/paper/2603.01601/) (2026)
- [Direct Reward Fine-Tuning on Poses for Single Image to 3D Human in the Wild](https://mlsanity.com/paper/2603.02619/) (2026)
- [PeCA: Palette Context Assisted Inference for Test-Time Paint-Bucket Colourisation on Animation Videos](https://mlsanity.com/paper/2608.00903/) (2026)

## Abstract (arXiv)

In the field of digital content creation, generating high-quality 3D characters from single images is challenging, especially given the complexities of various body poses and the issues of self-occlusion and pose ambiguity. In this paper, we present CharacterGen, a framework developed to efficiently generate 3D characters. CharacterGen introduces a streamlined generation pipeline along with an image-conditioned multi-view diffusion model. This model effectively calibrates input poses to a canonical form while retaining key attributes of the input image, thereby addressing the challenges posed by diverse poses. A transformer-based, generalizable sparse-view reconstruction model is the other core component of our approach, facilitating the creation of detailed 3D models from multi-view images. We also adopt a texture-back-projection strategy to produce high-quality texture maps. Additionally, we have curated a dataset of anime characters, rendered in multiple poses and views, to train and evaluate our model. Our approach has been thoroughly evaluated through quantitative and qualitative experiments, showing its proficiency in generating 3D characters with high-quality shapes and textures, ready for downstream applications such as rigging and animation.