mlsanityData to 14 Oct 2026

Part-X-MLLM: Part-aware 3D Multimodal Large Language Model

arXiv 2511.13647 ↗First posted 17 Nov 2025cs.CVcited by 7

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

We introduce Part-X-MLLM, a native 3D multimodal large language model that unifies diverse 3D tasks by formulating them as programs in a structured, executable grammar. Given an RGB point cloud and a natural language prompt, our model autoregressively generates a single, coherent token sequence encoding part-level bounding boxes, semantic descriptions, and edit commands. This structured output serves as a versatile interface to drive downstream geometry-aware modules for part-based generation and editing. By decoupling the symbolic planning from the geometric synthesis, our approach allows any compatible geometry engine to be controlled through a single, language-native frontend. We pre-train a dual-encoder architecture to disentangle structure from semantics and instruction-tune the model on a large-scale, part-centric dataset. Experiments demonstrate that our model excels at producing high-quality, structured plans, enabling state-of-the-art performance in grounded Q&A, compositional generation, and localized editing through one unified interface. Project page: https://chunshi.wang/Part-X-MLLM/

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 56 of its 122 references that are in the map. First posted on arXiv 17 Nov 2025.

Builds on

56 of 122 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and Understanding2 Jun
  2. 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence29 May
  3. 20253D-LLaVA: Towards Generalist 3D LMMs with Omni Superpoint Transformer2 Jan
  4. 2024Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understanding30 Nov
  5. 2024ShapeLLM: Universal 3D Object Understanding for Embodied Interaction27 Feb
  6. 2023GPT4Point: A Unified Framework for Point-Language Understanding and Generation5 Dec
  7. 2023LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning30 Nov

Other references · 49

  1. 2023Segment Anythingcited by 8,234
  2. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  3. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  4. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 770
  5. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 653
  6. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 484
  7. 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Modelscited by 458
  8. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 455
  9. 2025Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generationcited by 356
  10. 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 333
  11. 2023Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representationcited by 228
  12. 2023Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifierscited by 175
37 more
  1. 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 175
  2. 2025Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Materialcited by 165
  3. 2023Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenescited by 129
  4. 2025Hunyuan3D 2.5: Towards High-Fidelity 3D Assets Generation with Ultimate Detailscited by 125
  5. 2024CraftsMan3D: High-fidelity Mesh Generation with 3D Native Generation and Interactive Geometry Refinercited by 123
  6. 2023Vox-E: Text-guided Voxel Editing of 3D Objectscited by 115
  7. 2025Direct3D-S2: Gigascale 3D Generation Made Easy with Spatial Sparse Attentioncited by 110
  8. 2025Hi3DGen: High-fidelity 3D Geometry Generation from Images via Normal Bridgingcited by 99
  9. 2024SAMPart3D: Segment Any Part in 3D Objectscited by 97
  10. 2025PARTFIELD: Learning 3D Feature Fields for Part Segmentation and Beyondcited by 95
  11. 2024LLaMA-Mesh: Unifying 3D Mesh Generation with Language Modelscited by 91
  12. 2025Sparc3D: Sparse Representation and Construction for High-Resolution 3D Shapes Modelingcited by 89
  13. 2023FocalDreamer: Text-driven 3D Editing via Focal-fusion Assemblycited by 76
  14. 2025DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learningcited by 65
  15. 2024PartGen: Part-level 3D Generation and Reconstruction with Multi-View Diffusion Modelscited by 62
  16. 2025OmniPart: Part-Aware 3D Generation with Semantic Decoupling and Structural Cohesioncited by 61
  17. 2025HoloPart: Generative 3D Part Amodal Segmentationcited by 60
  18. 2024DreamReward: Text-to-3D Generation with Human Preferencecited by 56
  19. 2024Part123: Part-aware 3D Reconstruction from a Single-view Imagecited by 55
  20. 2025Efficient Part-level 3D Object Generation via Dual Volume Packingcited by 48
  21. 2024Generic 3D Diffusion Adapter Using Controlled Multi-View Editingcited by 47
  22. 2025VoxHammer: Training-Free Precise and Coherent 3D Editing in Native 3D Spacecited by 45
  23. 2025Ultra3D: Efficient and High-Fidelity 3D Generation with Part Attentioncited by 44
  24. 2025X-Part: high fidelity and structure coherent shape decompositioncited by 44
  25. 2025P3-SAM: Native 3D Part Segmentationcited by 43
  26. 2023SHAP-EDITOR: Instruction-guided Latent 3D Editing in Secondscited by 37
  27. 2024The Scene Language: Representing Scenes with Programs, Words, and Embeddingscited by 36
  28. 2025BANG: Dividing 3D Assets via Generative Exploded Dynamicscited by 35
  29. 2025AutoPartGen: Autogressive 3D Part Generation and Discoverycited by 33
  30. 2024Make-Your-3D: Fast and Consistent Subject-Driven 3D Content Generationcited by 32
  31. 2025NANO3D: A Training-Free Approach for Efficient 3D Editing Without Maskscited by 31
  32. 2024PrEditor3D: Fast and Precise 3D Shape Editingcited by 28
  33. 2024Frankenstein: Generating Semantic-Compositional 3D Scenes in One Tri-Planecited by 26
  34. 2024PhyCAGE: Physically Plausible Compositional 3D Asset Generation from a Single Imagecited by 19
  35. 2024PASTA: Controllable Part-Aware Shape Generation with Autoregressive Transformerscited by 15
  36. 2025Assembler: Scalable 3D Part Assembly via Anchor Point Diffusioncited by 12
  37. 2025LangScene-X: Reconstruct Generalizable 3D Language-Embedded Scenes with TriMap Video Diffusioncited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026ArtLLM: Generating Articulated Assets via 3D LLMcited by 8
  2. 2026PAR3D: A Unified 3D-MLLM with Part-Aware Representation for Scene Understandingcited by 1
  3. 2026PolyFlow: Continuous Topology Embedding Flow Matching for Artist-style Mesh Generationcited by 1
  4. 20263D-PLOT-LLM: Part-Level Object Tokens for 3D Large Language Modelscited by 0
  5. 2026Hunyuan3D-Buffalo 1.0: A Unified Multimodal Model for Scalable 3D Generation, Understanding, and Editingcited by 0
  6. 2026SCULPT: Subtractive Composition for 3D Part Generationcited by 0
  7. 2026Octrees as an Explicit 3D Languagecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 20243DBench: A Scalable 3D Benchmark and Instruction-Tuning Dataset
  2. 2026MegaParts: Scaling Part-Aware 3D Object Generation to 300 Parts via Token-Efficient Autoregressive Modeling
  3. 2026CG-MLLM: Captioning and Generating 3D content via Multi-modal Large Language Models
  4. 20253UR-LLM: An End-to-End Multimodal Large Language Model for 3D Scene Understanding
  5. 2026GAP-MLLM: Geometry-Aligned Pre-training for Activating 3D Spatial Perception in Multimodal Large Language Models
  6. 2025PiSA: A Self-Augmented Data Engine and Training Strategy for 3D Understanding with Large Models
  7. 2026EVA01: Unified Native 3D Understanding and Generation via Mixture-of-Transformers
  8. 20253DFroMLLM: 3D Prototype Generation only from Pretrained Multimodal LLMs