mlsanityData to 14 Oct 2026

DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learning

arXiv 2503.15265 ↗First posted 19 Mar 2025cs.CVcited by 65

On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment

also in 1 more
Abstract (arXiv)

Triangle meshes play a crucial role in 3D applications for efficient manipulation and rendering. While auto-regressive methods generate structured meshes by predicting discrete vertex tokens, they are often constrained by limited face counts and mesh incompleteness. To address these challenges, we propose DeepMesh, a framework that optimizes mesh generation through two key innovations: (1) an efficient pre-training strategy incorporating a novel tokenization algorithm, along with improvements in data curation and processing, and (2) the introduction of Reinforcement Learning (RL) into 3D mesh generation to achieve human preference alignment via Direct Preference Optimization (DPO). We design a scoring standard that combines human evaluation with 3D metrics to collect preference pairs for DPO, ensuring both visual appeal and geometric accuracy. Conditioned on point clouds and images, DeepMesh generates meshes with intricate details and precise topology, outperforming state-of-the-art methods in both precision and quality. Project page: https://zhaorw02.github.io/DeepMesh/

Key facts. Cited by 65 papers in the map (data through 14 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 93 of its 100 references that are in the map. First posted on arXiv 19 Mar 2025.

Builds on

93 of 100 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Aligning Modalities in Vision Large Language Models via Preference Fine-tuning18 Feb
  2. 2023Silkie: Preference Distillation for Large Visual Language Models17 Dec
  3. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization28 Nov

Other references · 90

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  4. 2017Neural Discrete Representation Learningcited by 4,712
  5. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  6. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  7. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  8. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  9. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  10. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
  11. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
  12. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 938
38 more
  1. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
  2. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 843
  3. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 774
  4. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 770
  5. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
  6. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 699
  7. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 689
  8. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 687
  9. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 645
  10. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  11. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 560
  12. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 521
  13. 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 463
  14. 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Modelscited by 458
  15. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 455
  16. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  17. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 378
  18. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 366
  19. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusioncited by 364
  20. 2024GS-LRM: Large Reconstruction Model for 3D Gaussian Splattingcited by 349
  21. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusioncited by 309
  22. 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 301
  23. 2023Text-to-3D using Gaussian Splattingcited by 299
  24. 2022Teaching language models to support answers with verified quotescited by 292
  25. 2023MeshGPT: Generating Triangle Meshes with Decoder-Only Transformerscited by 259
  26. 2023Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformerscited by 258
  27. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generationcited by 257
  28. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
  29. 2023DreamBooth3D: Subject-Driven Text-to-3D Generationcited by 235
  30. 2023Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representationcited by 228
  31. 2023MVImgNet: A Large-scale Dataset of Multi-view Imagescited by 220
  32. 2024CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Modelcited by 215
  33. 2023DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Modelcited by 214
  34. 2021PAConv: Position Adaptive Convolution with Dynamic Kernel Assembling on Point Cloudscited by 211
  35. 2024Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformercited by 210
  36. 2023MeshDiffusion: Score-based Generative 3D Mesh Modelingcited by 199
  37. 2022EGSDE: Unpaired Image-to-Image Translation via Energy-Guided Stochastic Differential Equationscited by 172
  38. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3Dcited by 168

50 most cited of 90 All 90 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 30; 2026: 35.

65 papers.

Sort:

By area:

  1. 2025PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformerscited by 84
  2. 2025OmniPart: Part-Aware 3D Generation with Semantic Decoupling and Structural Cohesioncited by 61
  3. 2025Efficient Part-level 3D Object Generation via Dual Volume Packingcited by 48
  4. 2025VoxHammer: Training-Free Precise and Coherent 3D Editing in Native 3D Spacecited by 45
  5. 2025Ultra3D: Efficient and High-Fidelity 3D Generation with Part Attentioncited by 44
  6. 2025ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and Understandingcited by 38
  7. 2025NANO3D: A Training-Free Approach for Efficient 3D Editing Without Maskscited by 31
  8. 2025Mesh-RFT: Enhancing Mesh Generation via Fine-grained Reinforcement Fine-Tuningcited by 22
  9. 2025MeshCoder: LLM-Powered Structured Mesh Code Generation from Point Cloudscited by 19
  10. 2025Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advancescited by 12
  11. 2025QuadGPT: Native Quadrilateral Mesh Generation with Autoregressive Modelscited by 12
  12. 2025MeshRipple: Structured Autoregressive Generation of Artist-Meshescited by 10
38 more
  1. 2025Are We Ready for RL in Text-to-3D Generation? A Progressive Investigationcited by 10
  2. 2025CoRe3D: Collaborative Reasoning as a Foundation for 3D Intelligencecited by 10
  3. 2026LATO: 3D Mesh Flow Matching with Structured TOpology Preserving LAtentscited by 10
  4. 2026SegviGen: Repurposing 3D Generative Model for Part Segmentationcited by 10
  5. 2025MeshMosaic: Scaling Artist Mesh Generation via Local-to-Global Assemblycited by 8
  6. 2025DreamCS: Geometry-Aware Text-to-3D Generation with Unpaired 3D Reward Supervisioncited by 7
  7. 2025Topology-Preserved Auto-regressive Mesh Generation in the Manner of Weaving Silkcited by 7
  8. 2025Part-X-MLLM: Part-aware 3D Multimodal Large Language Modelcited by 7
  9. 2026MeshFlow: Efficient Artistic Mesh Generation via MeshVAE and Flow-based Diffusion Transformercited by 7
  10. 2025XSpecMesh: Quality-Preserving Auto-Regressive Mesh Generation Acceleration via Multi-Head Speculative Decodingcited by 6
  11. 2025Vision-Language Models as Differentiable Semantic and Spatial Rewards for Text-to-3D Generationcited by 6
  12. 2026FACE: A Face-based Autoregressive Representation for High-Fidelity and Efficient Mesh Generationcited by 6
  13. 2026GEM: Generative Supervision Helps Embodied Intelligencecited by 6
  14. 2026LATO.2: Factorized 3D Mesh Generation with Vertex and Topology Flowcited by 6
  15. 2025Topology Sculptor, Shape Refiner: Discrete Diffusion Model for High-Fidelity 3D Meshes Generationcited by 5
  16. 2026RecGen3D: Reconstruction-Guided 3D Generation in a Shared Canonical Spacecited by 5
  17. 2025CHARM: Control-point-based 3D Anime Hairstyle Auto-Regressive Modelingcited by 4
  18. 2025PartDiffuser: Part-wise 3D Mesh Generation via Discrete Diffusioncited by 4
  19. 2026MeshFlow: Mesh Generation with Equivariant Flow Matchingcited by 4
  20. 2026Meshy T2: Fast Native Mesh Generation with Flow Matchingcited by 4
  21. 2025FlashMesh: Faster and Better Autoregressive Mesh Synthesis via Structured Speculationcited by 3
  22. 2025DragMesh: Interactive 3D Generation Made Easycited by 3
  23. 2026Mesh-Pro: Asynchronous Advantage-guided Ranking Preference Optimization for Artist-style Quadrilateral Mesh Generationcited by 3
  24. 2026PixARMesh: Autoregressive Mesh-Native Single-View Scene Reconstructioncited by 3
  25. 2026OpenWorldLib: A Unified Codebase and Definition of Advanced World Modelscited by 3
  26. 2026Strips as Tokens: Artist Mesh Generation with Native UV Segmentationcited by 3
  27. 2026Repurposing 3D Generative Model for Autoregressive Layout Generationcited by 3
  28. 2025SeamCrafter: Enhancing Mesh Seam Generation for Artist UV Unwrapping via Reinforcement Learningcited by 2
  29. 2025Photo3D: Advancing Photorealistic 3D Generation through Structure-Aligned Detail Enhancementcited by 2
  30. 2026AssetGen: Deployable 3D Asset Generation at Interactive Speedcited by 2
  31. 2026BrickAnything: Geometry-Conditioned Buildable Brick Generation with Structure-Aware Tokenizationcited by 2
  32. 2026MeshWeaver: Sparse-Voxel-Guided Surface Weaving for Autoregressive Mesh Generationcited by 2
  33. 2026TriFlow: Generating Artist-Like 3D Mesh Topology via Nearest-Vertex Vector Fieldscited by 2
  34. 2025Artist-Created Mesh Generation from Raw Observationcited by 1
  35. 2025Mesh RAG: Retrieval Augmentation for Autoregressive Mesh Generationcited by 1
  36. 2026QuadLink: Autoregressive Quad-Dominant Mesh Generation via Point-Relation Learningcited by 1
  37. 2026PolyFlow: Continuous Topology Embedding Flow Matching for Artist-style Mesh Generationcited by 1
  38. 2026Mesh BDF: Barycentric Dominance Field for 3D Native Mesh Generationcited by 1

50 most cited of 65 All 65 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2020Modeling 3D Shapes by Reinforcement Learning
  2. 2026VLM-Guided Group Preference Alignment for Diffusion-based Human Mesh Recovery
  3. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models
  4. 2026ActionMesh: Animated 3D Mesh Generation with Temporal 3D Diffusion
  5. 2025MeshCraft: Exploring Efficient and Controllable Mesh Generation with Flow-based DiTs
  6. 2024R2-Mesh: Reinforcement Learning Powered Mesh Reconstruction via Geometry and Appearance Refinement
  7. 2026Dmsh: A Multi-Agent Reinforcement Learning Framework for All-Quad Mesh Generation
  8. 2026DB-3DME: From Dataset to Benchmark for Human-aligned Automatic 3D Mesh Evaluation