mlsanityData to 7 Oct 2026

DreamReward: Text-to-3D Generation with Human Preference

arXiv 2403.14613 ↗First posted 21 Mar 2024cs.CV cs.CL cs.LGcited by 56

Not on the map. Why?

Abstract (arXiv)

3D content creation from text prompts has shown remarkable success recently. However, current text-to-3D methods often generate 3D results that do not align well with human preferences. In this paper, we present a comprehensive framework, coined DreamReward, to learn and improve text-to-3D models from human preference feedback. To begin with, we collect 25k expert comparisons based on a systematic annotation pipeline including rating and ranking. Then, we build Reward3D -- the first general-purpose text-to-3D human preference reward model to effectively encode human preferences. Building upon the 3D reward model, we finally perform theoretical analysis and present the Reward3D Feedback Learning (DreamFL), a direct tuning algorithm to optimize the multi-view diffusion models with a redefined scorer. Grounded by theoretical proof and extensive experiment comparisons, our DreamReward successfully generates high-fidelity and 3D consistent results with significant boosts in prompt alignment with human intention. Our results demonstrate the great potential for learning from human feedback to improve text-to-3D models.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). Builds on 49 of its 57 references that are in the map. First posted on arXiv 21 Mar 2024.

Builds on

49 of 57 references.

References · 49

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  10. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  11. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  12. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
37 more
  1. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  2. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  3. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  5. 2020Learning to summarize from human feedbackcited by 2,562
  6. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  7. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  8. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  9. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  10. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  11. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  12. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  13. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  14. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  15. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 767
  16. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
  17. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  18. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  19. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  20. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  21. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  22. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 431
  23. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  24. 2023GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splattingcited by 331
  25. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusioncited by 283
  26. 2023Using Human Feedback to Fine-tune Diffusion Models without Any Reward Modelcited by 246
  27. 2023Scalable 3D Captioning with Pretrained Modelscited by 201
  28. 20233DGen: Triplane Latent Diffusion for Textured Mesh Generationcited by 181
  29. 2023DreamEditor: Text-Driven 3D Scene Editing with Neural Fieldscited by 154
  30. 2024GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generationcited by 151
  31. 2023SweetDreamer: Aligning Geometric Priors in 2D Diffusion for Consistent Text-to-3Dcited by 134
  32. 2023HiFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidancecited by 109
  33. 2023Diffusion Models for Reinforcement Learning: A Surveycited by 84
  34. 2024Polos: Multimodal Metric Learning from Human Feedback for Image Captioningcited by 46
  35. 2023Sherpa3D: Boosting High-Fidelity Text-to-3D Generation via Coarse 3D Priorcited by 32
  36. 2023Text-to-3D Generation with Bidirectional Diffusion using both 2D and 3D priorscited by 24
  37. 2023Adversarial Score Distillation: When score distillation meets GANcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 12; 2025: 29; 2026: 15.

56 papers.

Sort:

By area:

  1. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
  2. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
  3. 2024DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusioncited by 139
  4. 2024Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Imagecited by 132
  5. 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 76
  6. 2025DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learningcited by 62
  7. 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 37
  8. 2026Lyra 2.0: Explorable Generative 3D Worldscited by 36
  9. 2024Alignment of Diffusion Models: Fundamentals, Challenges, and Futurecited by 34
  10. 2025ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and Understandingcited by 34
  11. 2024Baking Gaussian Splatting into Diffusion Denoiser for Fast and Scalable Single-stage Image-to-3D Generation and Reconstructioncited by 32
  12. 2025NANO3D: A Training-Free Approach for Efficient 3D Editing Without Maskscited by 29
38 more
  1. 2025Generating Physically Stable and Buildable Brick Structures from Textcited by 23
  2. 2025Mesh-RFT: Enhancing Mesh Generation via Fine-grained Reinforcement Fine-Tuningcited by 21
  3. 2024VividDreamer: Invariant Score Distillation For Hyper-Realistic Text-to-3D Generationcited by 15
  4. 2024A Survey On Text-to-3D Contents Generation In The Wildcited by 13
  5. 2025Smoothed Preference Optimization via ReNoise Inversion for Aligning Diffusion Models with Varied Human Preferencescited by 13
  6. 20253DGen-Bench: Comprehensive Benchmark Suite for 3D Generative Modelscited by 12
  7. 2025Efficient Autoregressive Shape Generation via Octree-Based Adaptive Tokenizationcited by 12
  8. 2025Nabla-R2D3: Effective and Efficient 3D Diffusion Alignment with 2D Rewardscited by 11
  9. 2025Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advancescited by 11
  10. 2025Omni-Reward: Towards Generalist Omni-Modal Reward Modeling with Free-Form Preferencescited by 11
  11. 2026One2Scene: Geometric Consistent Explorable 3D Scene Generation from a Single Imagecited by 10
  12. 2025CoRe3D: Collaborative Reasoning as a Foundation for 3D Intelligencecited by 9
  13. 2025Reinforcement Learning for Large Model: A Surveycited by 8
  14. 2025MeshRipple: Structured Autoregressive Generation of Artist-Meshescited by 8
  15. 2024Benchmarking and Learning Multi-Dimensional Quality Evaluator for Text-to-3D Generationcited by 7
  16. 2025DreamCS: Geometry-Aware Text-to-3D Generation with Unpaired 3D Reward Supervisioncited by 7
  17. 2026ReWorld: Multi-Dimensional Reward Modeling for Embodied World Modelscited by 7
  18. 2024MinD-3D++: Advancing fMRI-Based 3D Reconstruction with High-Quality Textured Mesh Generation and a Comprehensive Datasetcited by 6
  19. 2025Vision-Language Models as Differentiable Semantic and Spatial Rewards for Text-to-3D Generationcited by 6
  20. 2025Part-X-MLLM: Part-aware 3D Multimodal Large Language Modelcited by 6
  21. 2025SegmentDreamer: Towards High-fidelity Text-to-3D Synthesis with Segmented Consistency Trajectory Distillationcited by 5
  22. 2026Beyond the Dirac Delta: Mitigating Diversity Collapse in Reinforcement Fine-Tuning for Versatile Image Generationcited by 5
  23. 2025RewardSDS: Aligning Score Distillation via Reward-Weighted Samplingcited by 4
  24. 2025Text-to-3D Generation using Jensen-Shannon Score Distillationcited by 2
  25. 2025Refining Few-Step Text-to-Multiview Diffusion via Reinforcement Learningcited by 2
  26. 2025Towards Fine-Grained Text-to-3D Quality Assessment: A Benchmark and A Two-Stage Rank-Learning Metriccited by 2
  27. 2025Track, Inpaint, Resplat: Subject-driven 3D and 4D Generation with Progressive Texture Infillingcited by 2
  28. 2026Stroke3D: Lifting 2D strokes into rigged 3D model via latent diffusion modelscited by 2
  29. 2026Preference Score Distillation: Leveraging 2D Rewards to Align Text-to-3D Generation with Human Preferencecited by 2
  30. 2025Towards More Accurate Personalized Image Generation: Addressing Overfitting and Evaluation Biascited by 1
  31. 2025Bridging Geometry-Coherent Text-to-3D Generation with Multi-View Diffusion Priors and Gaussian Splattingcited by 1
  32. 2025Point Cloud Compression and Objective Quality Assessment: A Surveycited by 1
  33. 2025Let Language Constrain Geometry: Vision-Language Models as Semantic and Spatial Critics for 3D Generationcited by 1
  34. 2026PolyFlow: Continuous Topology Embedding Flow Matching for Artist-style Mesh Generationcited by 1
  35. 2026iARCS: Iterative Agentic RL for Controllable 3D Scene Generationcited by 1
  36. 2026FMReward: Aligning and Evaluating Audio-Driven 3D Facial Animation with Human Preferencescited by 1
  37. 2024DiffusionGuard: A Robust Defense Against Malicious Diffusion-based Image Editingcited by 0
  38. 2025End-to-End Fine-Tuning of 3D Texture Generation using Differentiable Rewardscited by 0

50 most cited of 56 All 56 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimization
  2. 2025Text-driven 3D Human Generation via Contrastive Preference Optimization
  3. 2024MVReward: Better Aligning and Evaluating Multi-View Diffusion Models with Human Preferences
  4. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation
  5. 2024Make-Your-3D: Fast and Consistent Subject-Driven 3D Content Generation
  6. 2024VP3D: Unleashing 2D Visual Prompt for Text-to-3D Generation
  7. 2023Debiasing Scores and Prompts of 2D Diffusion for View-consistent Text-to-3D Generation
  8. 2025Dive3D: Diverse Distillation-based Text-to-3D Generation via Score Implicit Matching