# MeshGen: Generating PBR Textured Mesh with Render-Enhanced Auto-Encoder and Generative Data Augmentation

arXiv: https://arxiv.org/abs/2505.04656 · HTML: https://mlsanity.com/paper/2505.04656/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 May 2025.

## Built on by (most cited first, 5 of 5)

- [Ultra3D: Efficient and High-Fidelity 3D Generation with Part Attention](https://mlsanity.com/paper/2507.17745/) (2025)
- [Video Perception Models for 3D Scene Synthesis](https://mlsanity.com/paper/2506.20601/) (2025)
- [NeAR: Coupled Neural Asset-Renderer Stack](https://mlsanity.com/paper/2511.18600/) (2025)
- [Sat2City v2: Native 3D City Asset Generation from a Single Satellite Image](https://mlsanity.com/paper/2606.24138/) (2026)
- [MatLat: Material Latent Space for PBR Texture Generation](https://mlsanity.com/paper/2512.17302/) (2025)

## Abstract (arXiv)

In this paper, we introduce MeshGen, an advanced image-to-3D pipeline that generates high-quality 3D meshes with detailed geometry and physically based rendering (PBR) textures. Addressing the challenges faced by existing 3D native diffusion models, such as suboptimal auto-encoder performance, limited controllability, poor generalization, and inconsistent image-based PBR texturing, MeshGen employs several key innovations to overcome these limitations. We pioneer a render-enhanced point-to-shape auto-encoder that compresses meshes into a compact latent space by designing perceptual optimization with ray-based regularization. This ensures that the 3D shapes are accurately represented and reconstructed to preserve geometric details within the latent space. To address data scarcity and image-shape misalignment, we further propose geometric augmentation and generative rendering augmentation techniques, which enhance the model's controllability and generalization ability, allowing it to perform well even with limited public datasets. For the texture generation, MeshGen employs a reference attention-based multi-view ControlNet for consistent appearance synthesis. This is further complemented by our multi-view PBR decomposer that estimates PBR components and a UV inpainter that fills invisible areas, ensuring a seamless and consistent texture across the 3D mesh. Our extensive experiments demonstrate that MeshGen largely outperforms previous methods in both shape and texture generation, setting a new standard for the quality of 3D meshes generated with PBR textures. See our code at https://github.com/heheyas/MeshGen, project page https://heheyas.github.io/MeshGen