mlsanityData to 14 Oct 2026

LLM-Guided Material Inference for 3D Point Clouds

arXiv 2512.03237 ↗First posted 2 Dec 2025cs.CV cs.GRcited by 0

Not on the map. Why?

Abstract (arXiv)

Most existing 3D shape datasets and models focus solely on geometry, overlooking the material properties that determine how objects appear. We introduce a two-stage large language model (LLM) based method for inferring material composition directly from 3D point clouds with coarse segmentations. Our key insight is to decouple reasoning about what an object is from what it is made of. In the first stage, an LLM predicts the object's semantic; in the second stage, it assigns plausible materials to each geometric segment, conditioned on the inferred semantics. Both stages operate in a zero-shot manner, without task-specific training. Because existing datasets lack reliable material annotations, we evaluate our method using an LLM-as-a-Judge implemented in DeepEval. Across 1,000 shapes from Fusion/ABS and ShapeNet, our method achieves high semantic and material plausibility. These results demonstrate that language models can serve as general-purpose priors for bridging geometric reasoning and material understanding in 3D data.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 38 references that are in the map. First posted on arXiv 2 Dec 2025.

Builds on

29 of 38 references.

References · 29

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  3. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  4. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,223
  5. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 748
  6. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
  7. 2023DETRs Beat YOLOs on Real-time Object Detectioncited by 533
  8. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 526
  9. 2022PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategiescited by 436
  10. 2021PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformerscited by 286
  11. 2022ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understandingcited by 267
  12. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
17 more
  1. 2023OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understandingcited by 184
  2. 2017Learning to Predict Indoor Illumination from a Single Imagecited by 161
  3. 2023LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an Agentcited by 135
  4. 2023Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenescited by 129
  5. 2018Single-Image SVBRDF Capture with a Rendering-Aware Deep Networkcited by 116
  6. 2023Neural Kernel Surface Reconstructioncited by 104
  7. 2020ParSeNet: A Parametric Surface Fitting Network for 3D Point Cloudscited by 96
  8. 2023OneFormer3D: One Transformer for Unified Point Cloud Segmentationcited by 95
  9. 2018Supervised Fitting of Geometric Primitives to 3D Point Cloudscited by 91
  10. 2014Material Recognition in the Wild with the Materials in Context Databasecited by 79
  11. 2021JoinABLe: Learning Bottom-up Assembly of Parametric CAD Jointscited by 78
  12. 2025CAD-Llama: Leveraging Large Language Models for Computer-Aided Design Parametric 3D Model Generationcited by 67
  13. 2025HoLa: B-Rep Generation using a Holistic Latent Representationcited by 30
  14. 2024Make-it-Real: Unleashing Large Multimodal Model for Painting 3D Objects with Realistic Materialscited by 17
  15. 2018Recognizing Material Properties from Imagescited by 14
  16. 2025Better STEP, a format and dataset for boundary representationcited by 2
  17. 2024PS-CAD: Local Geometry Guidance via Prompting and Selection for CAD Reconstructioncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Single Object: Learning 3D Relations with Large Language Models
  2. 2024MaterialSeg3D: Segmenting Dense Materials from 2D Priors for 3D Assets
  3. 2025Can Large Language Models Identify Materials from Radar Signals? ↗
  4. 2024Reason3D: Searching and Reasoning 3D Segmentation via Large Language Model
  5. 2026PartMat: Material-Aware 3D Part Decomposition with a Single Global Latent
  6. 2026Imagine3D-LLM: Teaching MLLMs to Imagine 3D Scenes Before Answering
  7. 20263D-PLOT-LLM: Part-Level Object Tokens for 3D Large Language Models
  8. 2023Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Following