mlsanityData to 13 Oct 2026

Shape: A Self-Supervised 3D Geometry Foundation Model for Industrial CAD Analysis

arXiv 2604.22826 ↗First posted 19 Apr 2026cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Industrial CAD workflows require robust, generalizable 3D geometric representations supporting accuracy and explainability. We introduce Shape, a self-supervised foundation model converting surface meshes into dense per-token embeddings. Shape combines a structured 3D latent grid, a multi-scale geometry-aware tokenizer (MAGNO) with cross-attention, and a transformer processor using grouped-query attention and RMSNorm. A learned reconstruction prior enables per-region attribution for explainable predictions. Pretraining uses masked-token reconstruction of normalized geometry statistics and multi-resolution contrastive consistency. The 10.9M-parameter backbone is pretrained on 61,052 CAD meshes from Thingi10K, MFCAD, and Fusion360. On a held-out split of 2,983 meshes, Shape achieves reconstruction R2 = 0.729 and 98.1% top-1 retrieval under the Wang-Isola protocol, with near-zero reconstruction train/val gap (contrastive scores use a larger evaluation pool). A 2x2 ablation on loss type and target-space normalization shows per-dimension normalization is critical: without it, performance collapses (R2 < 0.14, top-1 < 88%); with it, both losses succeed (R2 > 0.70, top-1 > 96%). Smooth-L1 offers secondary stability. Code, embeddings, and an interactive demo are released at https://github.com/simd-ai/shape.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 25 of its 31 references that are in the map. First posted on arXiv 19 Apr 2026.

Builds on

25 of 31 references.

References · 25

  1. 2017Attention Is All You Needcited by 60,106
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,043
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,009
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,857
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,712
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,364
  8. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,231
  9. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,628
  10. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,389
  11. 2015Fast R-CNNcited by 2,933
  12. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,285
13 more
  1. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,529
  2. 2019Root Mean Square Layer Normalizationcited by 1,449
  3. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,397
  4. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,323
  5. 2020Neural Operator: Graph Kernel Network for Partial Differential Equationscited by 728
  6. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 648
  7. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 524
  8. 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 457
  9. 2018ABC: A Big CAD Model Dataset For Geometric Deep Learningcited by 341
  10. 2022Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-trainingcited by 263
  11. 2016Thingi10K: A Dataset of 10,000 3D-Printing Modelscited by 182
  12. 2022From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagationcited by 141
  13. 2025Geometry Aware Operator Transformer as an Efficient and Accurate Neural Surrogate for PDEs on Arbitrary Domainscited by 59

Built on by

Not cited yet.