# DAE-Net: Deforming Auto-Encoder for fine-grained shape co-segmentation

arXiv: https://arxiv.org/abs/2311.13125 · HTML: https://mlsanity.com/paper/2311.13125/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 78 references that are in the map.
- First posted on arXiv 22 Nov 2023.

## Builds on (most cited first, 30 of 53)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [Grounded Language-Image Pre-training](https://mlsanity.com/paper/2112.03857/) (2021)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [Language-driven Semantic Segmentation](https://mlsanity.com/paper/2201.03546/) (2022)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [In-Place Scene Labelling and Understanding with Implicit Scene Representation](https://mlsanity.com/paper/2103.15875/) (2021)
- [Decomposing NeRF for Editing via Feature Field Distillation](https://mlsanity.com/paper/2205.15585/) (2022)
- [GLIPv2: Unifying Localization and Vision-Language Understanding](https://mlsanity.com/paper/2206.05836/) (2022)
- [Zero-Shot Semantic Segmentation](https://mlsanity.com/paper/1906.00817/) (2019)
- [Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representation](https://mlsanity.com/paper/2205.04334/) (2022)
- [BSP-Net: Generating Compact Meshes via Binary Space Partitioning](https://mlsanity.com/paper/1911.06971/) (2019)
- [Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representations](https://mlsanity.com/paper/2209.03494/) (2022)
- [Learning Shape Abstractions by Assembling Volumetric Primitives](https://mlsanity.com/paper/1612.00404/) (2016)
- [PLA: Language-Driven Open-Vocabulary 3D Scene Understanding](https://mlsanity.com/paper/2211.16312/) (2022)
- [CvxNet: Learnable Convex Decomposition](https://mlsanity.com/paper/1909.05736/) (2019)
- [Panoptic Lifting for 3D Scene Understanding with Neural Fields](https://mlsanity.com/paper/2212.09802/) (2022)
- [Neural Volumes: Learning Dynamic Renderable Volumes from Images](https://mlsanity.com/paper/1906.07751/) (2019)
- [CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memory](https://mlsanity.com/paper/2210.05663/) (2022)
- [Panoptic NeRF: 3D-to-2D Label Transfer for Panoptic Urban Scene Segmentation](https://mlsanity.com/paper/2203.15224/) (2022)
- [Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids](https://mlsanity.com/paper/1904.09970/) (2019)
- [Deformed Implicit Field: Modeling 3D Shapes with Learned Dense Correspondence](https://mlsanity.com/paper/2011.13650/) (2020)

## Built on by (most cited first, 7 of 7)

- [Hierarchical Neural Semantic Representation for 3D Semantic Correspondence](https://mlsanity.com/paper/2509.17431/) (2025)
- [Beyond Complete Shapes: A Benchmark for Quantitative Evaluation of 3D Shape Surface Matching Algorithms](https://mlsanity.com/paper/2411.03511/) (2024)
- [PartSDF: Part-Based Implicit Neural Representation for Composite 3D Shape Parametrization and Optimization](https://mlsanity.com/paper/2502.12985/) (2025)
- [Self-supervised Learning of Hybrid Part-aware 3D Representations of 2D Gaussians and Superquadrics](https://mlsanity.com/paper/2408.10789/) (2024)
- [GenAnalysis: Joint Shape Analysis by Learning Man-Made Shape Generators with Deformation Regularizations](https://mlsanity.com/paper/2503.00807/) (2025)
- [SAR2Struct: Extracting 3D Semantic Structural Representation of Aircraft Targets from Single-View SAR Image](https://mlsanity.com/paper/2506.06757/) (2025)
- [Hierarchical Transformers for Unsupervised 3D Shape Abstraction](https://mlsanity.com/paper/2510.27088/) (2025)

## Abstract (arXiv)

We present an unsupervised 3D shape co-segmentation method which learns a set of deformable part templates from a shape collection. To accommodate structural variations in the collection, our network composes each shape by a selected subset of template parts which are affine-transformed. To maximize the expressive power of the part templates, we introduce a per-part deformation network to enable the modeling of diverse parts with substantial geometry variations, while imposing constraints on the deformation capacity to ensure fidelity to the originally represented parts. We also propose a training scheme to effectively overcome local minima. Architecturally, our network is a branched autoencoder, with a CNN encoder taking a voxel shape as input and producing per-part transformation matrices, latent codes, and part existence scores, and the decoder outputting point occupancies to define the reconstruction loss. Our network, coined DAE-Net for Deforming Auto-Encoder, can achieve unsupervised 3D shape co-segmentation that yields fine-grained, compact, and meaningful parts that are consistent across diverse shapes. We conduct extensive experiments on the ShapeNet Part dataset, DFAUST, and an animal subset of Objaverse to show superior performance over prior methods. Code and data are available at https://github.com/czq142857/DAE-Net.