mlsanityData to 14 Oct 2026

Text-guided 3D Human Generation from 2D Collections

arXiv 2305.14312 ↗First posted 23 May 2023cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

3D human modeling has been widely used for engaging interaction in gaming, film, and animation. The customization of these characters is crucial for creativity and scalability, which highlights the importance of controllability. In this work, we introduce Text-guided 3D Human Generation (textttT3H), where a model is to generate a 3D human, guided by the fashion description. There are two goals: 1) the 3D human should render articulately, and 2) its outfit is controlled by the given text. To address this textttT3H task, we propose Compositional Cross-modal Human (CCH). CCH adopts cross-modal attention to fuse compositional human rendering with the extracted fashion semantics. Each human body part perceives relevant textual guidance as its visual patterns. We incorporate the human prior and semantic discrimination to enhance 3D geometry transformation and fine-grained consistency, enabling it to learn from 2D collections for data efficiency. We conduct evaluations on DeepFashion and SHHQ with diverse fashion attributes covering the shape, fabric, and color of upper and lower clothing. Extensive experiments demonstrate that CCH achieves superior results for textttT3H with high efficiency.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 78 of its 92 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

78 of 92 references.

References · 78

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  6. 2022Generative Adversarial Networkscited by 5,627
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  8. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  9. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  10. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  11. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  12. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
38 more
  1. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,238
  2. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,964
  3. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  4. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  5. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  6. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,162
  7. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  8. 2021Volume Rendering of Neural Implicit Surfacescited by 996
  9. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 780
  10. 2016Generative Adversarial Text to Image Synthesiscited by 735
  11. 2020Implicit Geometric Regularization for Learning Shapescited by 710
  12. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  13. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 695
  14. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 694
  15. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 689
  16. 2018Which Training Methods for GANs do actually Converge?cited by 679
  17. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  18. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  19. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 627
  20. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  21. 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 537
  22. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  23. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  24. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 488
  25. 2022HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Videocited by 483
  26. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  27. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 388
  28. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 370
  29. 2021Animatable Neural Radiance Fields for Modeling Dynamic Human Bodiescited by 361
  30. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 343
  31. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 334
  32. 2023TEXTure: Text-Guided Texturing of 3D Shapescited by 304
  33. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  34. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  35. 2021Mixture of Volumetric Primitives for Efficient Neural Renderingcited by 277
  36. 2020PolyGen: An Autoregressive Generative Model of 3D Meshescited by 269
  37. 2021A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Posecited by 233
  38. 2020PaMIR: Parametric Model-Conditioned Implicit Representation for Image-based Human Reconstructioncited by 231

50 most cited of 78 All 78 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 2.

4 papers.

  1. 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
  2. 2024SemanticHuman-HD: High-Resolution Semantic Disentangled 3D Human Generationcited by 3
  3. 2025PARTE: Part-Guided Texturing for 3D Human Reconstruction from a Single Imagecited by 2
  4. 2025HuGeDiff: 3D Human Generation via Diffusion with Gaussian Splattingcited by 1