mlsanityData to 14 Oct 2026

DreamCS: Geometry-Aware Text-to-3D Generation with Unpaired 3D Reward Supervision

arXiv 2506.09814 ↗First posted 11 Jun 2025cs.CVcited by 7

On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment

Abstract (arXiv)

While text-to-3D generation has attracted growing interest, existing methods often struggle to produce 3D assets that align well with human preferences. Current preference alignment techniques for 3D content typically rely on hardly-collected preference-paired multi-view 2D images to train 2D reward models, when then guide 3D generation -- leading to geometric artifacts due to their inherent 2D bias. To address these limitations, we construct 3D-MeshPref, the first large-scale unpaired 3D preference dataset, featuring diverse 3D meshes annotated by a large language model and refined by human evaluators. We then develop RewardCS, the first reward model trained directly on unpaired 3D-MeshPref data using a novel Cauchy-Schwarz divergence objective, enabling effective learning of human-aligned 3D geometric preferences without requiring paired comparisons. Building on this, we propose DreamCS, a unified framework that integrates RewardCS into text-to-3D pipelines -- enhancing both implicit and explicit 3D generation with human preference feedback. Extensive experiments show DreamCS outperforms prior methods, producing 3D assets that are both geometrically faithful and human-preferred. Code and models will be released publicly.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 52 of its 74 references that are in the map. First posted on arXiv 11 Jun 2025.

Builds on

52 of 74 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learning19 Mar
  2. 2025DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimization5 Feb
  3. 2025Improving Video Generation with Human Feedback23 Jan
  4. 2024VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generation30 Dec
  5. 2024GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation8 Jan
  6. 2023Rich Human Feedback for Text-to-Image Generation15 Dec
  7. 2023Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model22 Nov

Other references · 45

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2017Deep reinforcement learning from human preferencescited by 3,912
  5. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  6. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  7. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  8. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  9. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
  10. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
  11. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  12. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
33 more
  1. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 963
  2. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 938
  3. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 770
  4. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 687
  5. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  6. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 521
  7. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 460
  8. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 397
  9. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 385
  10. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 366
  11. 2016Kernel Mean Embedding of Distributions: A Review and Beyondcited by 364
  12. 2023Recognize Anything: A Strong Image Tagging Modelcited by 309
  13. 2024Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMscited by 273
  14. 2023Principled Reinforcement Learning with Human Feedback from Pairwise or K-wise Comparisonscited by 247
  15. 2023Scalable 3D Captioning with Pretrained Modelscited by 203
  16. 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Priorcited by 158
  17. 2023TextMesh: Generation of Realistic 3D Meshes From Text Promptscited by 135
  18. 2021Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillationcited by 134
  19. 2023HiFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidancecited by 110
  20. 2024MeshAnything V2: Artist-Created Mesh Generation With Adjacent Mesh Tokenizationcited by 104
  21. 2024LLaMA-Mesh: Unifying 3D Mesh Generation with Language Modelscited by 91
  22. 2019On a generalization of the Jensen-Shannon divergence and the JS-symmetrization of distances relying on abstract meanscited by 61
  23. 2024Consistent3D: Towards Consistent High-Fidelity Text-to-3D Generation with Deterministic Sampling Priorcited by 60
  24. 2024DreamReward: Text-to-3D Generation with Human Preferencecited by 56
  25. 2023DreamControl: Control-Based Text-to-3D Generation with 3D Self-Priorcited by 39
  26. 2022MeshMAE: Masked Autoencoders for 3D Mesh Data Analysiscited by 31
  27. 2023Carve3D: Improving Multi-view Reconstruction Consistency for Diffusion Models with RL Finetuningcited by 22
  28. 2023MVControl: Adding Conditional Control to Multi-view Diffusion for Controllable Text-to-3D Generationcited by 8
  29. 2025Distributional Vision-Language Alignment by Cauchy-Schwarz Divergencecited by 7
  30. 2024Domain Adaptation with Cauchy-Schwarz Divergencecited by 6
  31. 2024Geometry-Aware Score Distillation via 3D Consistent Noising and Gradient Consistency Modelingcited by 6
  32. 2024Deep Geometric Moments Promote Shape Consistency in Text-to-3D Generationcited by 5
  33. 2025HPS: Hard Preference Sampling for Human Preference Alignmentcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.

Similar, not cited

Close in wording, no citation either way

  1. 20253DGen-Bench: Comprehensive Benchmark Suite for 3D Generative Models
  2. 2025Text-driven 3D Human Generation via Contrastive Preference Optimization
  3. 2025Are We Ready for RL in Text-to-3D Generation? A Progressive Investigation
  4. 2025End-to-End Fine-Tuning of 3D Texture Generation using Differentiable Rewards
  5. 2025Let Language Constrain Geometry: Vision-Language Models as Semantic and Spatial Critics for 3D Generation
  6. 2026ELSA3D: Elastic Semantic Anchoring for Unified 3D Understanding and Generation
  7. 2024MVReward: Better Aligning and Evaluating Multi-View Diffusion Models with Human Preferences
  8. 2025Dive3D: Diverse Distillation-based Text-to-3D Generation via Score Implicit Matching