mlsanityData to 14 Oct 2026

PAR3D: A Unified 3D-MLLM with Part-Aware Representation for Scene Understanding

arXiv 2606.06485 ↗First posted 4 Jun 2026cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Recent advances in 3D multimodal large language models (3D-MLLMs) have enabled unified solutions for 3D scene understanding tasks, including visual question answering, captioning, and referring segmentation. However, existing 3D-MLLMs remain largely object-centric, limiting their ability to model fine-grained part structures that are essential for embodied interaction with 3D environments. In this work, we present PAR3D, a unified part-aware 3D-MLLM framework that enables models to understand, reason about, and ground both objects and their parts in 3D scenes. To enable training and evaluation of part-aware 3D scene understanding, we introduce ScenePart, a synthetic 3D scene dataset with part-level annotations and language instructions. We further develop Part-Aware 3D Representation Learning to enrich 3D visual representations with fine-grained part-level semantics, and propose Hierarchical Segmentation Query Generation to ground part targets via hierarchical object-part queries. Extensive experiments show that our method substantially improves part-level question answering and referring segmentation, while also achieving strong performance across object-level vision-language tasks.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 62 of its 73 references that are in the map. First posted on arXiv 4 Jun 2026.

Builds on

62 of 73 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025Part-X-MLLM: Part-aware 3D Multimodal Large Language Model17 Nov
  2. 2025LEO-VL: Efficient Scene Representation for Scalable 3D Vision-Language Learning11 Jun
  3. 20253D-LLaVA: Towards Generalist 3D LMMs with Omni Superpoint Transformer2 Jan
  4. 2024Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understanding30 Nov
  5. 2024LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness26 Sep
  6. 2024Reason3D: Searching and Reasoning 3D Segmentation via Large Language Model27 May
  7. 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Models16 May
  8. 2024MiniGPT-3D: Efficiently Aligning 3D Point Clouds with Large Language Models using 2D Priors2 May
  9. 2023PartDistill: 3D Shape Part Segmentation by Vision-Language Model Distillation7 Dec
  10. 2023GPT4Point: A Unified Framework for Point-Language Understanding and Generation5 Dec
  11. 2023LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning30 Nov

Other references · 51

  1. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  2. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  5. 2025Qwen3-VL Technical Reportcited by 2,793
  6. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
  7. 2020Point Transformercited by 1,470
  8. 2018PointCNN: Convolution On X-Transformed Pointscited by 764
  9. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 595
  10. 2023Point Transformer V3: Simpler, Faster, Strongercited by 589
  11. 2023LERF: Language Embedded Radiance Fieldscited by 543
  12. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 526
38 more
  1. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 454
  2. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 452
  3. 2023LangSplat: 3D Language Gaussian Splattingcited by 415
  4. 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 380
  5. 2023An Embodied Generalist Agent in 3D Worldcited by 365
  6. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 346
  7. 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 333
  8. 2022SQA3D: Situated Question Answering in 3D Scenescited by 314
  9. 20203D-FUTURE: 3D Furniture shape with TextUREcited by 311
  10. 2020Scan2Cap: Context-aware Dense Captioning in RGB-D Scanscited by 218
  11. 20233D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignmentcited by 204
  12. 2019RIO: 3D Object Instance Re-Localization in Changing Indoor Environmentscited by 188
  13. 2021Where2Act: From Pixels to Actions for Articulated 3D Objectscited by 182
  14. 2023Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifierscited by 175
  15. 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 175
  16. 2023Multi3DRefer: Grounding Text Description to Multiple 3D Objectscited by 146
  17. 2022GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation via Generalizable and Actionable Partscited by 139
  18. 2022PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Modelscited by 118
  19. 2024Grounded 3D-LLM with Referent Tokenscited by 115
  20. 2025Sonata: Self-Supervised Learning of Reliable Point Representationscited by 110
  21. 2024SAMPart3D: Segment Any Part in 3D Objectscited by 97
  22. 2023End-to-End 3D Dense Captioning with Vote2Cap-DETRcited by 78
  23. 2022UniT3D: A Unified Transformer for 3D Dense Captioning and Visual Groundingcited by 69
  24. 2024GOI: Find 3D Gaussians of Interest with an Optimizable Open-vocabulary Semantic-space Hyperplanecited by 53
  25. 2024Point-SAM: Promptable 3D Segmentation Model for Point Cloudscited by 50
  26. 2024ScanReason: Empowering 3D Visual Grounding with Reasoning Capabilitiescited by 47
  27. 2025Concerto: Joint 2D-3D Self-Supervised Learning Emerges Spatial Representationscited by 45
  28. 2025P3-SAM: Native 3D Part Segmentationcited by 43
  29. 2025How to Enable LLM with 3D Capacity? A Survey of Spatial Reasoning in LLMcited by 42
  30. 20233D-STMN: Dependency-Driven Superpoint-Text Matching Network for End-to-End 3D Referring Expression Segmentationcited by 36
  31. 2024SegPoint: Segment Any Point Cloud via Large Language Modelcited by 35
  32. 2024Mixed Diffusion for 3D Indoor Scene Synthesiscited by 32
  33. 2024Find Any Part in 3Dcited by 32
  34. 20243DGraphLLM: Combining Semantic Graphs and Large Language Models for 3D Scene Understandingcited by 27
  35. 2025Drag Your Gaussian: Effective Drag-Based Editing with Score Distillation for 3D Gaussian Splattingcited by 22
  36. 20233DCoMPaT++: An improved Large-scale 3D Vision Dataset for Compositional Recognitioncited by 21
  37. 2026Utonia: Toward One Encoder for All Point Cloudscited by 21
  38. 2025Training-Free Hierarchical Scene Understanding for Gaussian Splatting with Superpoint Graphscited by 10

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Distill What RGB Can Recover: Privileged 3D Evidence for RGB-Only Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Text-Scene: A Scene-to-Language Parsing Framework for 3D Scene Understanding
  2. 20253DRS: MLLMs Need 3D-Aware Representation Supervision for Scene Understanding
  3. 2025HMR3D: Hierarchical Multimodal Representation for 3D Scene Understanding with Large Vision-Language Model
  4. 2025MLLM-For3D: Adapting Multimodal Large Language Model for 3D Reasoning Segmentation
  5. 2026SceneBench: A Hierarchical Benchmark for Vision-Language Understanding of 3D Scenes
  6. 20253UR-LLM: An End-to-End Multimodal Large Language Model for 3D Scene Understanding
  7. 20263D-PLOT-LLM: Part-Level Object Tokens for 3D Large Language Models
  8. 2024PARIS3D: Reasoning-based 3D Part Segmentation Using Large Multimodal Model