# 3DCoMPaT++: An improved Large-scale 3D Vision Dataset for Compositional Recognition

arXiv: https://arxiv.org/abs/2310.18511 · HTML: https://mlsanity.com/paper/2310.18511/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 42 references that are in the map.
- First posted on arXiv 27 Oct 2023.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Objaverse-XL: A Universe of 10M+ 3D Objects](https://mlsanity.com/paper/2307.05663/) (2023)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items](https://mlsanity.com/paper/2204.11918/) (2022)
- [LION: Latent Point Diffusion Models for 3D Shape Generation](https://mlsanity.com/paper/2210.06978/) (2022)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies](https://mlsanity.com/paper/2206.04670/) (2022)
- [Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework](https://mlsanity.com/paper/2202.07123/) (2022)
- [ABO: Dataset and Benchmarks for Real-World 3D Object Understanding](https://mlsanity.com/paper/2110.06199/) (2021)
- [OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation](https://mlsanity.com/paper/2301.07525/) (2023)
- [3D-FUTURE: 3D Furniture shape with TextURE](https://mlsanity.com/paper/2009.09633/) (2020)
- [Walk in the Cloud: Learning Curves for Point Clouds Shape Analysis](https://mlsanity.com/paper/2105.01288/) (2021)
- [Bidirectional Projection Network for Cross Dimension Scene Understanding](https://mlsanity.com/paper/2103.14326/) (2021)
- [Grounded Situation Recognition](https://mlsanity.com/paper/2003.12058/) (2020)
- [OpenRooms: An End-to-End Open Framework for Photorealistic Indoor Scene Datasets](https://mlsanity.com/paper/2007.12868/) (2020)
- [Advanced Feature Learning on Point Clouds using Multi-resolution Features and Learnable Pooling](https://mlsanity.com/paper/2205.09962/) (2022)
- [Learning Material-Aware Local Descriptors for 3D Shapes](https://mlsanity.com/paper/1810.08729/) (2018)
- [HAL3D: Hierarchical Active Learning for Fine-Grained 3D Part Labeling](https://mlsanity.com/paper/2301.10460/) (2023)

## Built on by (most cited first, 21 of 21)

- [PartSAM: A Scalable Promptable Part Segmentation Model Trained on Native 3D Data](https://mlsanity.com/paper/2509.21965/) (2025)
- [Kestrel: 3D Multimodal LLM for Part-Aware Grounded Description](https://mlsanity.com/paper/2405.18937/) (2024)
- [HyperSDFusion: Bridging Hierarchical Structures in Language and Geometry for Enhanced 3D Text2Shape Generation](https://mlsanity.com/paper/2403.00372/) (2024)
- [ParSEL: Parameterized Shape Editing with Language](https://mlsanity.com/paper/2405.20319/) (2024)
- [SOPHY: Learning to Generate Simulation-Ready Objects with Physical Materials](https://mlsanity.com/paper/2504.12684/) (2025)
- [Learning Part-Aware Dense 3D Feature Field for Generalizable Articulated Object Manipulation](https://mlsanity.com/paper/2602.14193/) (2026)
- [TexPro: Text-guided PBR Texturing with Procedural Material Modeling](https://mlsanity.com/paper/2410.15891/) (2024)
- [3DCoMPaT200: Language-Grounded Compositional Understanding of Parts and Materials of 3D Shapes](https://mlsanity.com/paper/2501.06785/) (2025)
- [MultiView-Bench: A Diagnostic Benchmark for World-Centric Multi-View Integration in VLMs](https://mlsanity.com/paper/2607.08970/) (2026)
- [GeoMask3D: Geometrically Informed Mask Selection for Self-Supervised Point Cloud Learning in 3D](https://mlsanity.com/paper/2405.12419/) (2024)
- [Name That Part: 3D Part Segmentation and Naming](https://mlsanity.com/paper/2512.18003/) (2025)
- [LychSim: A Controllable and Interactive Simulation Framework for Vision Research](https://mlsanity.com/paper/2605.12449/) (2026)
- [MatCLIP: Light- and Shape-Insensitive Assignment of PBR Material Models](https://mlsanity.com/paper/2501.15981/) (2025)
- [PAR3D: A Unified 3D-MLLM with Part-Aware Representation for Scene Understanding](https://mlsanity.com/paper/2606.06485/) (2026)
- [FllumaOne: A Code-Native Multimodal CAD Dataset with Executable Programs and Kernel-Validated Feature Histories](https://mlsanity.com/paper/2606.17696/) (2026)
- [Adaptive Volumetric Mechanical Property Fields Invariant to Resolution](https://mlsanity.com/paper/2606.18231/) (2026)
- [PRISM: Probabilistic Representation for Integrated Shape Modeling and Generation](https://mlsanity.com/paper/2504.04454/) (2025)
- [OMGTex: One-stage Multi-style Facial Texture Reconstruction without Geometry Guidance](https://mlsanity.com/paper/2605.25778/) (2026)
- [3D-PLOT-LLM: Part-Level Object Tokens for 3D Large Language Models](https://mlsanity.com/paper/2606.19828/) (2026)
- [PART: Learning 3D Part Assembly and Retrieval with Transformers](https://mlsanity.com/paper/2609.19872/) (2026)
- [Organize Primitives into Semantic Parts: Reinforcement Reasoning for 3D Segmentation](https://mlsanity.com/paper/2610.04602/) (2026)

## Abstract (arXiv)

In this work, we present 3DCoMPaT++, a multimodal 2D/3D dataset with 160 million rendered views of more than 10 million stylized 3D shapes carefully annotated at the part-instance level, alongside matching RGB point clouds, 3D textured meshes, depth maps, and segmentation masks. 3DCoMPaT++ covers 41 shape categories, 275 fine-grained part categories, and 293 fine-grained material classes that can be compositionally applied to parts of 3D objects. We render a subset of one million stylized shapes from four equally spaced views as well as four randomized views, leading to a total of 160 million renderings. Parts are segmented at the instance level, with coarse-grained and fine-grained semantic levels. We introduce a new task, called Grounded CoMPaT Recognition (GCR), to collectively recognize and ground compositions of materials on parts of 3D objects. Additionally, we report the outcomes of a data challenge organized at CVPR2023, showcasing the winning method's utilization of a modified PointNet++ model trained on 6D inputs, and exploring alternative techniques for GCR enhancement. We hope our work will help ease future research on compositional 3D Vision.