Text-driven 3D Human Generation via Contrastive Preference Optimization
On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment
Abstract (arXiv)
Recent advances in Score Distillation Sampling (SDS) have improved 3D human generation from textual descriptions. However, existing methods still face challenges in accurately aligning 3D models with long and complex textual inputs. To address this challenge, we propose a novel framework that introduces contrastive preferences, where human-level preference models, guided by both positive and negative prompts, assist SDS for improved alignment. Specifically, we design a preference optimization module that integrates multiple models to comprehensively capture the full range of textual features. Furthermore, we introduce a negation preference module to mitigate over-optimization of irrelevant details by leveraging static-dynamic negation prompts, effectively preventing ``reward hacking". Extensive experiments demonstrate that our method achieves state-of-the-art results, significantly enhancing texture realism and visual alignment with textual descriptions, particularly for long and complex inputs.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 39 of its 44 references that are in the map. First posted on arXiv 13 Feb 2025.
Builds on
39 of 44 references.
Closest ancestors · 2
Same direction, newest first
- 2024Improving Long-Text Alignment for Text-to-Image Diffusion Models15 Oct
- 2024Multimodal Large Language Model is a Human-Aligned Annotator for Text-to-Image Generation23 Apr
Other references · 37
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
- 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
- 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
- 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
- 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
- 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
- 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
- 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
25 more
- 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
- 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
- 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
- 2018DensePose: Dense Human Pose Estimation In The Wildcited by 573
- 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
- 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
- 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
- 2023DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Modelscited by 154
- 2023TADA! Text to Animatable Digital Avatarscited by 109
- 2021imGHUM: Implicit Generative Models of 3D Human Shape and Articulated Posecited by 102
- 2023DreamHuman: Animatable 3D Avatars from Textcited by 102
- 2024LongAlign: A Recipe for Long Context Alignment of Large Language Modelscited by 102
- 2023HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splattingcited by 89
- 2023HumanNorm: Learning Normal Diffusion Model for High-quality and Realistic 3D Human Generationcited by 88
- 2023AvatarCraft: Transforming Text into Neural Human Avatars with Parameterized Shape and Pose Controlcited by 84
- 2023DreamWaltz: Make a Scene with Complex 3D Animatable Avatarscited by 82
- 2023GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learningcited by 63
- 20243D Gaussian Splatting with Deferred Reflectioncited by 61
- 2023AvatarVerse: High-quality & Stable 3D Avatar Creation from Text and Posecited by 60
- 2023AvatarBooth: High-Quality and Customizable 3D Human Avatar Generationcited by 37
- 2024WE-GS: An In-the-wild Efficient 3D Gaussian Representation for Unconstrained Photo Collectionscited by 33
- 2024DreamWaltz-G: Expressive 3D Gaussian Avatars from Skeleton-Guided 2D Diffusioncited by 12
- 2024X-Oscar: A Progressive Framework for High-quality Text-guided 3D Animatable Avatar Generationcited by 5
- 2024DreamMapping: High-Fidelity Text-to-3D Generation via Variational Distribution Mappingcited by 5
- 2024MVGaussian: High-Fidelity text-to-3D Content Generation with Multi-View Guidance and Surface Densificationcited by 5
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimization
- 2026Preference Score Distillation: Leveraging 2D Rewards to Align Text-to-3D Generation with Human Preference
- 2024DreamReward: Text-to-3D Generation with Human Preference
- 20253DGen-Bench: Comprehensive Benchmark Suite for 3D Generative Models
- 2025DreamCS: Geometry-Aware Text-to-3D Generation with Unpaired 3D Reward Supervision
- 2024GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation
- 2025Dive3D: Diverse Distillation-based Text-to-3D Generation via Score Implicit Matching
- 2024VP3D: Unleashing 2D Visual Prompt for Text-to-3D Generation