Mesh-RFT: Enhancing Mesh Generation via Fine-grained Reinforcement Fine-Tuning
On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment
also in 1 more
Abstract (arXiv)
Existing pretrained models for 3D mesh generation often suffer from data biases and produce low-quality results, while global reinforcement learning (RL) methods rely on object-level rewards that struggle to capture local structure details. To address these challenges, we present Mesh-RFT, a novel fine-grained reinforcement fine-tuning framework that employs Masked Direct Preference Optimization (M-DPO) to enable localized refinement via quality-aware face masking. To facilitate efficient quality evaluation, we introduce an objective topology-aware scoring system to evaluate geometric integrity and topological regularity at both object and face levels through two metrics: Boundary Edge Ratio (BER) and Topology Score (TS). By integrating these metrics into a fine-grained RL strategy, Mesh-RFT becomes the first method to optimize mesh quality at the granularity of individual faces, resolving localized errors while preserving global coherence. Experiment results show that our M-DPO approach reduces Hausdorff Distance (HD) by 24.6% and improves Topology Score (TS) by 3.8% over pre-trained models, while outperforming global DPO methods with a 17.4% HD reduction and 4.9% TS gain. These results demonstrate Mesh-RFT's ability to improve geometric integrity and topological regularity, achieving new state-of-the-art performance in production-ready mesh generation. Project Page: https://hitcslj.github.io/mesh-rft/.
Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 64 of its 71 references that are in the map. First posted on arXiv 22 May 2025.
Builds on
64 of 71 references.
Closest ancestors · 2
Same direction, newest first
- 2025DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learning19 Mar
- 2024Aligning Modalities in Vision Large Language Models via Preference Fine-tuning18 Feb
Other references · 62
- 2017Attention Is All You Needcited by 60,361
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
- 2017Neural Discrete Representation Learningcited by 4,712
- 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
- 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
- 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
38 more
- 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
- 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
- 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
- 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
- 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
- 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 843
- 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 774
- 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
- 2021Diffusion Probabilistic Models for 3D Point Cloud Generationcited by 735
- 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 687
- 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
- 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 463
- 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Modelscited by 458
- 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 455
- 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 433
- 2025Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generationcited by 356
- 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 333
- 20203D-FUTURE: 3D Furniture shape with TextUREcited by 311
- 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 301
- 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 288
- 2025VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Taskscited by 285
- 2020PolyGen: An Autoregressive Generative Model of 3D Meshescited by 269
- 2023MeshGPT: Generating Triangle Meshes with Decoder-Only Transformerscited by 259
- 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
- 2023Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representationcited by 228
- 2024CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Modelcited by 215
- 2025TripoSG: High-Fidelity 3D Shape Synthesis using Large-Scale Rectified Flow Modelscited by 213
- 2024Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformercited by 210
- 2024MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformerscited by 144
- 2023Locally Attentional SDF Diffusion for Controllable 3D Shape Generationcited by 130
- 2024CraftsMan3D: High-fidelity Mesh Generation with 3D Native Generation and Interactive Geometry Refinercited by 123
- 2024MeshAnything V2: Artist-Created Mesh Generation With Adjacent Mesh Tokenizationcited by 104
- 2024MeshXL: Neural Coordinate Field for Generative 3D Foundation Modelscited by 103
- 2021Hierarchical Transformers Are More Efficient Language Modelscited by 102
- 2024Meshtron: High-Fidelity, Artist-Like 3D Mesh Generation at Scalecited by 89
- 2024EdgeRunner: Auto-regressive Auto-encoder for Artistic Mesh Generationcited by 87
- 2025Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured 3D Assetscited by 87
- 2025SparseFlex: High-Resolution and Arbitrary-Topology 3D Shape Modelingcited by 79
50 most cited of 62 All 62 (JSON)
Built on by
Citing papers per year: 2025: 7; 2026: 15.
22 papers.
Sort:
By area:
- 2025ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and Understandingcited by 38
- 2025Hunyuan3D Studio: End-to-End AI Pipeline for Game-Ready 3D Asset Generationcited by 13
- 2025Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advancescited by 12
- 2025QuadGPT: Native Quadrilateral Mesh Generation with Autoregressive Modelscited by 12
- 2025MeshRipple: Structured Autoregressive Generation of Artist-Meshescited by 10
- 2026LATO: 3D Mesh Flow Matching with Structured TOpology Preserving LAtentscited by 10
- 2025XSpecMesh: Quality-Preserving Auto-Regressive Mesh Generation Acceleration via Multi-Head Speculative Decodingcited by 6
- 2026FACE: A Face-based Autoregressive Representation for High-Fidelity and Efficient Mesh Generationcited by 6
- 2026LATO.2: Factorized 3D Mesh Generation with Vertex and Topology Flowcited by 6
- 2025FlashMesh: Faster and Better Autoregressive Mesh Synthesis via Structured Speculationcited by 3
- 2026Mesh-Pro: Asynchronous Advantage-guided Ranking Preference Optimization for Artist-style Quadrilateral Mesh Generationcited by 3
- 2026PixARMesh: Autoregressive Mesh-Native Single-View Scene Reconstructioncited by 3
10 more
- 2026Strips as Tokens: Artist Mesh Generation with Native UV Segmentationcited by 3
- 2026HiFi-Mesh: High-Fidelity Efficient 3D Mesh Generation via Compact Autoregressive Dependencecited by 2
- 2026BrickAnything: Geometry-Conditioned Buildable Brick Generation with Structure-Aware Tokenizationcited by 2
- 2026MeshWeaver: Sparse-Voxel-Guided Surface Weaving for Autoregressive Mesh Generationcited by 2
- 2026QuadLink: Autoregressive Quad-Dominant Mesh Generation via Point-Relation Learningcited by 1
- 2026SQuadGen: Generating Simple Quad Layouts via Chart Distance Fieldscited by 0
- 2026Mesh-RL: Coupled subgrid reinforcement learningcited by 0
- 2026Sculpting NeRF Geometry: Human-Preference Fine-Tuning of a 3D-Aware Face GANcited by 0
- 2026MeshOctave: Vertex Split-and-Rewire Cascades for Native Mesh Generationcited by 0
- 2026Q-MAGIC: Quad-only Mesh Autoregressive Generation via Singularity-Aware Circular Relative Offset Tokenizationcited by 0
- 2026Q-MAGIC: Quad-only Mesh Autoregressive Generation via Singularity-Aware Circular Relative Offset Tokenization14 Oct 2026
- 2026MeshOctave: Vertex Split-and-Rewire Cascades for Native Mesh Generation30 Sep 2026
- 2026LATO.2: Factorized 3D Mesh Generation with Vertex and Topology Flow12 Jul 2026
- 2026Sculpting NeRF Geometry: Human-Preference Fine-Tuning of a 3D-Aware Face GAN25 Jun 2026
- 2026Mesh-RL: Coupled subgrid reinforcement learning24 Jun 2026
- 2026MeshWeaver: Sparse-Voxel-Guided Surface Weaving for Autoregressive Mesh Generation3 Jun 2026
- 2026BrickAnything: Geometry-Conditioned Buildable Brick Generation with Structure-Aware Tokenization25 May 2026
- 2026QuadLink: Autoregressive Quad-Dominant Mesh Generation via Point-Relation Learning16 May 2026
- 2026SQuadGen: Generating Simple Quad Layouts via Chart Distance Fields30 Apr 2026
- 2026Strips as Tokens: Artist Mesh Generation with Native UV Segmentation10 Apr 2026
- 2026LATO: 3D Mesh Flow Matching with Structured TOpology Preserving LAtents6 Mar 2026
- 2026PixARMesh: Autoregressive Mesh-Native Single-View Scene Reconstruction6 Mar 2026
10 more
- 2026FACE: A Face-based Autoregressive Representation for High-Fidelity and Efficient Mesh Generation2 Mar 2026
- 2026Mesh-Pro: Asynchronous Advantage-guided Ranking Preference Optimization for Artist-style Quadrilateral Mesh Generation28 Feb 2026
- 2026HiFi-Mesh: High-Fidelity Efficient 3D Mesh Generation via Compact Autoregressive Dependence29 Jan 2026
- 2025MeshRipple: Structured Autoregressive Generation of Artist-Meshes8 Dec 2025
- 2025FlashMesh: Faster and Better Autoregressive Mesh Synthesis via Structured Speculation19 Nov 2025
- 2025QuadGPT: Native Quadrilateral Mesh Generation with Autoregressive Models25 Sep 2025
- 2025Hunyuan3D Studio: End-to-End AI Pipeline for Game-Ready 3D Asset Generation16 Sep 2025
- 2025Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advances14 Aug 2025
- 2025XSpecMesh: Quality-Preserving Auto-Regressive Mesh Generation Acceleration via Multi-Head Speculative Decoding31 Jul 2025
- 2025ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and Understanding2 Jun 2025
- 2025Hunyuan3D Studio: End-to-End AI Pipeline for Game-Ready 3D Asset Generationcited by 13
- 2025QuadGPT: Native Quadrilateral Mesh Generation with Autoregressive Modelscited by 12
- 2025MeshRipple: Structured Autoregressive Generation of Artist-Meshescited by 10
- 2026FACE: A Face-based Autoregressive Representation for High-Fidelity and Efficient Mesh Generationcited by 6
- 2025FlashMesh: Faster and Better Autoregressive Mesh Synthesis via Structured Speculationcited by 3
- 2026PixARMesh: Autoregressive Mesh-Native Single-View Scene Reconstructioncited by 3
- 2026Strips as Tokens: Artist Mesh Generation with Native UV Segmentationcited by 3
- 2026HiFi-Mesh: High-Fidelity Efficient 3D Mesh Generation via Compact Autoregressive Dependencecited by 2
- 2026BrickAnything: Geometry-Conditioned Buildable Brick Generation with Structure-Aware Tokenizationcited by 2
- 2026MeshWeaver: Sparse-Voxel-Guided Surface Weaving for Autoregressive Mesh Generationcited by 2
- 2026QuadLink: Autoregressive Quad-Dominant Mesh Generation via Point-Relation Learningcited by 1
- 2026SQuadGen: Generating Simple Quad Layouts via Chart Distance Fieldscited by 0
12 most cited of 15 All 15 (JSON)
- 2026Q-MAGIC: Quad-only Mesh Autoregressive Generation via Singularity-Aware Circular Relative Offset Tokenization14 Oct 2026
- 2026MeshOctave: Vertex Split-and-Rewire Cascades for Native Mesh Generation30 Sep 2026
- 2026Sculpting NeRF Geometry: Human-Preference Fine-Tuning of a 3D-Aware Face GAN25 Jun 2026
- 2026MeshWeaver: Sparse-Voxel-Guided Surface Weaving for Autoregressive Mesh Generation3 Jun 2026
- 2026BrickAnything: Geometry-Conditioned Buildable Brick Generation with Structure-Aware Tokenization25 May 2026
- 2026QuadLink: Autoregressive Quad-Dominant Mesh Generation via Point-Relation Learning16 May 2026
- 2026SQuadGen: Generating Simple Quad Layouts via Chart Distance Fields30 Apr 2026
- 2026Strips as Tokens: Artist Mesh Generation with Native UV Segmentation10 Apr 2026
- 2026PixARMesh: Autoregressive Mesh-Native Single-View Scene Reconstruction6 Mar 2026
- 2026FACE: A Face-based Autoregressive Representation for High-Fidelity and Efficient Mesh Generation2 Mar 2026
- 2026HiFi-Mesh: High-Fidelity Efficient 3D Mesh Generation via Compact Autoregressive Dependence29 Jan 2026
- 2025MeshRipple: Structured Autoregressive Generation of Artist-Meshes8 Dec 2025
12 newest of 15 All 15 (JSON)
Similar, not cited
Close in wording, no citation either way
- 2024R2-Mesh: Reinforcement Learning Powered Mesh Reconstruction via Geometry and Appearance Refinement
- 2026Dmsh: A Multi-Agent Reinforcement Learning Framework for All-Quad Mesh Generation
- 2025End-to-End Fine-Tuning of 3D Texture Generation using Differentiable Rewards
- 2025SeamCrafter: Enhancing Mesh Seam Generation for Artist UV Unwrapping via Reinforcement Learning
- 2026Flow3D-OPD: Multi-Teacher On-Policy Distillation for 3D Geometry Generation with Flow-Matching Diffusion Transformer
- 2026Meshy T2: Fast Native Mesh Generation with Flow Matching
- 2026Playing to Par: Reinforcement Learning for Provably Optimal Quadrilateral Block Decompositions
- 2024GNNRL-Smoothing: A Prior-Free Reinforcement Learning Model for Mesh Smoothing