mlsanityData to 14 Oct 2026

ARKit LabelMaker: A New Scale for Indoor 3D Scene Understanding

arXiv 2410.13924 ↗First posted 17 Oct 2024cs.CV cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Neural network performance scales with both model size and data volume, as shown in both language and image processing. This requires scaling-friendly architectures and large datasets. While transformers have been adapted for 3D vision, a `GPT-moment' remains elusive due to limited training data. We introduce ARKit LabelMaker, a large-scale real-world 3D dataset with dense semantic annotation that is more than three times larger than prior largest dataset. Specifically, we extend ARKitScenes with automatically generated dense 3D labels using an extended LabelMaker pipeline, tailored for large-scale pre-training. Training on our dataset improves accuracy across architectures, achieving state-of-the-art 3D semantic segmentation scores on ScanNet and ScanNet200, with notable gains on tail classes. Our code is available at https://labelmaker.org and our dataset at https://huggingface.co/datasets/labelmaker/arkit_labelmaker.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 41 of its 55 references that are in the map. First posted on arXiv 17 Oct 2024.

Builds on

41 of 55 references.

References · 41

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2023Segment Anythingcited by 8,234
  6. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  7. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  8. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  9. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  10. 2022Emergent Abilities of Large Language Modelscited by 2,512
  11. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  12. 2020Point Transformercited by 1,470
29 more
  1. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  2. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
  3. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,205
  4. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 904
  5. 2018PointCNN: Convolution On X-Transformed Pointscited by 764
  6. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
  7. 2023Point Transformer V3: Simpler, Faster, Strongercited by 589
  8. 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 565
  9. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 559
  10. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 454
  11. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 406
  12. 2022Point Transformer V2: Grouped Vector Attention and Partition-based Poolingcited by 388
  13. 2017Large-scale Point Cloud Semantic Segmentation with Superpoint Graphscited by 385
  14. 2019Structured3D: A Large Photo-realistic Dataset for Structured 3D Modelingcited by 286
  15. 2018SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filterscited by 261
  16. 2022Language-Grounded Indoor 3D Semantic Segmentation in the Wildcited by 246
  17. 2018Point Convolutional Neural Networks by Extension Operatorscited by 154
  18. 2021Mix3D: Out-of-Context Data Augmentation for 3D Scenescited by 126
  19. 2021Fast Point Transformercited by 104
  20. 2023Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Trainingcited by 86
  21. 2024Improving 2D Feature Representations by 3D-Aware Fine-Tuningcited by 71
  22. 2024OpenNeRF: Open Set 3D Neural Scene Segmentation with Pixel-Wise Features and Rendered Novel Viewscited by 69
  23. 2023PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigmcited by 56
  24. 2023AGILE3D: Attention Guided Interactive Multi-object 3D Segmentationcited by 33
  25. 2024Search3D: Hierarchical Open-Vocabulary 3D Segmentationcited by 31
  26. 2024P2P-Bridge: Diffusion Bridges for 3D Point Cloud Denoisingcited by 18
  27. 2023LABELMAKER: Automatic Semantic Label Generation from RGB-D Trajectoriescited by 14
  28. 2023Self-supervised Pre-training with Masked Shape Prediction for 3D Scene Understandingcited by 13
  29. 2024OpenDAS: Open-Vocabulary Domain Adaptation for 2D and 3D Segmentationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 2.

7 papers.

  1. 2025Open-Vocabulary Functional 3D Scene Graphs for Real-World Indoor Spacescited by 54
  2. 2025SLAG: Scalable Language-Augmented Gaussian Splattingcited by 4
  3. 2026Volume Transformer: Revisiting Vanilla Transformers for 3D Scene Understandingcited by 4
  4. 2025OpenCity3D: What do Vision-Language Models know about Urban Environments?cited by 3
  5. 2025Leveraging Automatic CAD Annotations for Supervised Learning in 3D Scene Understandingcited by 2
  6. 2025HouseLayout3D: A Benchmark and Training-Free Baseline for 3D Layout Estimation in the Wildcited by 1
  7. 2026Resolving Primitive-Sharing Ambiguity in Long-Tailed TLS-Based Industrial MEP Point Cloud Segmentation via Spatial Context Constraintscited by 1