UniMesh: Unifying 3D Mesh Understanding and Generation
Not on the map. Why?
Abstract (arXiv)
Recent advances in 3D vision have led to specialized models for either 3D understanding (e.g., shape classification, segmentation, reconstruction) or 3D generation (e.g., synthesis, completion, and editing). However, these tasks are often tackled in isolation, resulting in fragmented architectures and representations that hinder knowledge transfer and holistic scene modeling. To address these challenges, we propose UniMesh, a unified framework that jointly learns 3D generation and understanding within a single architecture. First, we introduce a novel Mesh Head that acts as a cross model interface, bridging diffusion based image generation with implicit shape decoders. Second, we develop Chain of Mesh (CoM), a geometric instantiation of iterative reasoning that enables user driven semantic mesh editing through a closed loop latent, prompting, and re generation cycle. Third, we incorporate a self reflection mechanism based on an Actor Evaluator Self reflection triad to diagnose and correct failures in high level tasks like 3D captioning. Experimental results demonstrate that UniMesh not only achieves competitive performance on standard benchmarks but also unlocks novel capabilities in iterative editing and mutual enhancement between generation and understanding. Code: https://github.com/AIGeeksGroup/UniMesh. Website: https://aigeeksgroup.github.io/UniMesh.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 61 of its 75 references that are in the map. First posted on arXiv 19 Apr 2026.
Builds on
61 of 75 references.
References · 61
- 2017Attention Is All You Needcited by 59,932
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
38 more
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
- 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
- 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
- 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 559
- 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
- 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
- 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusioncited by 364
- 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
- 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 298
- 2024TripoSR: Fast 3D Object Reconstruction from a Single Imagecited by 227
- 2023MVImgNet: A Large-scale Dataset of Multi-view Imagescited by 219
- 2023Scalable 3D Captioning with Pretrained Modelscited by 201
- 2024LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awarenesscited by 178
- 2020SqueezeSegV3: Spatially-Adaptive Convolution for Efficient Point-Cloud Segmentationcited by 170
- 2025Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Materialcited by 157
- 2024Motion Mamba: Efficient and Long Sequence Motion Generationcited by 111
- 2014From Captions to Visual Concepts and Backcited by 109
- 2024LN3DIFF++: Scalable Latent Neural Fields Diffusion for Speedy 3D Generationcited by 91
- 2024SF3D: Stable Fast 3D Mesh Reconstruction with UV-unwrapping and Illumination Disentanglementcited by 65
- 2024VFusion3D: Learning Scalable 3D Generative Models from Video Diffusion Modelscited by 56
- 20253D-R1: Enhancing Reasoning in 3D VLMs for Unified Scene Understandingcited by 47
- 2021Learning general and distinctive 3D local deep descriptors for point cloud registrationcited by 41
- 2025VolSplat: Rethinking Feed-Forward 3D Gaussian Splatting with Voxel-Aligned Predictioncited by 41
- 2025Motion Anything: Any to Motion Generationcited by 40
- 2025Nav-R1: Reasoning and Navigation in Embodied Scenescited by 37
- 2024View Selection for 3D Captioning via Diffusion Rankingcited by 31
- 2025ZPressor: Bottleneck-Aware Compression for Scalable Feed-Forward 3DGScited by 29
- 2025Unleashing Vecset Diffusion Model for Fast Shape Generationcited by 25
- 2024InfiniMotion: Mamba Boosts Memory in Transformer for Arbitrary Long Motion Generationcited by 23
- 2026LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Modelscited by 20
- 2024Motion Avatar: Generate Human and Animal Avatars with Arbitrary Motioncited by 19
- 2024KMM: Key Frame Mask Mamba for Extended Motion Generationcited by 16
- 2025DC-Scene: Data-Centric Learning for 3D Scene Understandingcited by 15
- 2025DriveGen3D: Boosting Feed-Forward Driving Scene Generation with Efficient Video Diffusioncited by 14
50 most cited of 61 All 61 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025UniUGG: Unified 3D Understanding and Generation via Geometric-Semantic Encoding
- 2026Know3D: Prompting 3D Generation with Knowledge from Vision-Language Models
- 2026Hunyuan3D-Buffalo 1.0: A Unified Multimodal Model for Scalable 3D Generation, Understanding, and Editing
- 2025PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers
- 2026ROAD: Reciprocal-Objective Alignment of Discriminative Semantics for 3D Shape Generation
- 2026Imagine3D-LLM: Teaching MLLMs to Imagine 3D Scenes Before Answering
- 2023Uni3DL: Unified Model for 3D and Language Understanding
- 2022Semantic Abstraction: Open-World 3D Scene Understanding from 2D Vision-Language Models