DreamCS: Geometry-Aware Text-to-3D Generation with Unpaired 3D Reward Supervision
On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment
Abstract (arXiv)
While text-to-3D generation has attracted growing interest, existing methods often struggle to produce 3D assets that align well with human preferences. Current preference alignment techniques for 3D content typically rely on hardly-collected preference-paired multi-view 2D images to train 2D reward models, when then guide 3D generation -- leading to geometric artifacts due to their inherent 2D bias. To address these limitations, we construct 3D-MeshPref, the first large-scale unpaired 3D preference dataset, featuring diverse 3D meshes annotated by a large language model and refined by human evaluators. We then develop RewardCS, the first reward model trained directly on unpaired 3D-MeshPref data using a novel Cauchy-Schwarz divergence objective, enabling effective learning of human-aligned 3D geometric preferences without requiring paired comparisons. Building on this, we propose DreamCS, a unified framework that integrates RewardCS into text-to-3D pipelines -- enhancing both implicit and explicit 3D generation with human preference feedback. Extensive experiments show DreamCS outperforms prior methods, producing 3D assets that are both geometrically faithful and human-preferred. Code and models will be released publicly.
Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 52 of its 74 references that are in the map. First posted on arXiv 11 Jun 2025.
Builds on
52 of 74 references.
Closest ancestors · 7
Same direction, newest first
- 2025DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learning19 Mar
- 2025DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimization5 Feb
- 2025Improving Video Generation with Human Feedback23 Jan
- 2024VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generation30 Dec
- 2024GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation8 Jan
- 2023Rich Human Feedback for Text-to-Image Generation15 Dec
- 2023Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model22 Nov
Other references · 45
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2017Deep reinforcement learning from human preferencescited by 3,912
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
- 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
- 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
- 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
- 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
- 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
- 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
- 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
33 more
- 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 963
- 2023MVDream: Multi-view Diffusion for 3D Generationcited by 938
- 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 770
- 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 687
- 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
- 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 521
- 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 460
- 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 397
- 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 385
- 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 366
- 2016Kernel Mean Embedding of Distributions: A Review and Beyondcited by 364
- 2023Recognize Anything: A Strong Image Tagging Modelcited by 309
- 2024Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMscited by 273
- 2023Principled Reinforcement Learning with Human Feedback from Pairwise or K-wise Comparisonscited by 247
- 2023Scalable 3D Captioning with Pretrained Modelscited by 203
- 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Priorcited by 158
- 2023TextMesh: Generation of Realistic 3D Meshes From Text Promptscited by 135
- 2021Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillationcited by 134
- 2023HiFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidancecited by 110
- 2024MeshAnything V2: Artist-Created Mesh Generation With Adjacent Mesh Tokenizationcited by 104
- 2024LLaMA-Mesh: Unifying 3D Mesh Generation with Language Modelscited by 91
- 2019On a generalization of the Jensen-Shannon divergence and the JS-symmetrization of distances relying on abstract meanscited by 61
- 2024Consistent3D: Towards Consistent High-Fidelity Text-to-3D Generation with Deterministic Sampling Priorcited by 60
- 2024DreamReward: Text-to-3D Generation with Human Preferencecited by 56
- 2023DreamControl: Control-Based Text-to-3D Generation with 3D Self-Priorcited by 39
- 2022MeshMAE: Masked Autoencoders for 3D Mesh Data Analysiscited by 31
- 2023Carve3D: Improving Multi-view Reconstruction Consistency for Diffusion Models with RL Finetuningcited by 22
- 2023MVControl: Adding Conditional Control to Multi-view Diffusion for Controllable Text-to-3D Generationcited by 8
- 2025Distributional Vision-Language Alignment by Cauchy-Schwarz Divergencecited by 7
- 2024Domain Adaptation with Cauchy-Schwarz Divergencecited by 6
- 2024Geometry-Aware Score Distillation via 3D Consistent Noising and Gradient Consistency Modelingcited by 6
- 2024Deep Geometric Moments Promote Shape Consistency in Text-to-3D Generationcited by 5
- 2025HPS: Hard Preference Sampling for Human Preference Alignmentcited by 2
Built on by
Citing papers per year: 2025: 3; 2026: 4.
7 papers.
- 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
- 2025Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advancescited by 12
- 2025Reinforcement Learning for Large Model: A Surveycited by 9
- 2026Preference Score Distillation: Leveraging 2D Rewards to Align Text-to-3D Generation with Human Preferencecited by 2
- 2026Block3D: Efficient Text-to-3D Generation via Block-Wise Diffusioncited by 2
- 2026iARCS: Iterative Agentic RL for Controllable 3D Scene Generationcited by 1
- 2026Sculpting NeRF Geometry: Human-Preference Fine-Tuning of a 3D-Aware Face GANcited by 0
- 2026Block3D: Efficient Text-to-3D Generation via Block-Wise Diffusion20 Aug 2026
- 2026iARCS: Iterative Agentic RL for Controllable 3D Scene Generation6 Aug 2026
- 2026Sculpting NeRF Geometry: Human-Preference Fine-Tuning of a 3D-Aware Face GAN25 Jun 2026
- 2026Preference Score Distillation: Leveraging 2D Rewards to Align Text-to-3D Generation with Human Preference2 Mar 2026
- 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Survey2 Sep 2025
- 2025Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advances14 Aug 2025
- 2025Reinforcement Learning for Large Model: A Survey11 Aug 2025
Similar, not cited
Close in wording, no citation either way
- 20253DGen-Bench: Comprehensive Benchmark Suite for 3D Generative Models
- 2025Text-driven 3D Human Generation via Contrastive Preference Optimization
- 2025Are We Ready for RL in Text-to-3D Generation? A Progressive Investigation
- 2025End-to-End Fine-Tuning of 3D Texture Generation using Differentiable Rewards
- 2025Let Language Constrain Geometry: Vision-Language Models as Semantic and Spatial Critics for 3D Generation
- 2026ELSA3D: Elastic Semantic Anchoring for Unified 3D Understanding and Generation
- 2024MVReward: Better Aligning and Evaluating Multi-View Diffusion Models with Human Preferences
- 2025Dive3D: Diverse Distillation-based Text-to-3D Generation via Score Implicit Matching