# Leveraging SE(3) Equivariance for Learning 3D Geometric Shape Assembly

arXiv: https://arxiv.org/abs/2309.06810 · HTML: https://mlsanity.com/paper/2309.06810/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 39 references that are in the map.
- First posted on arXiv 13 Sep 2023.

## Builds on (most cited first, 30 of 31)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds](https://mlsanity.com/paper/1802.08219/) (2018)
- [SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks](https://mlsanity.com/paper/2006.10503/) (2020)
- [3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data](https://mlsanity.com/paper/1807.02547/) (2018)
- [Vector Neurons: A General Framework for SO(3)-Equivariant Networks](https://mlsanity.com/paper/2104.12229/) (2021)
- [Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation](https://mlsanity.com/paper/2112.05124/) (2021)
- [Visual Translation Embedding Network for Visual Relation Detection](https://mlsanity.com/paper/1702.08319/) (2017)
- [Equivariant Point Network for 3D Point Cloud Analysis](https://mlsanity.com/paper/2103.14147/) (2021)
- [JoinABLe: Learning Bottom-up Assembly of Parametric CAD Joints](https://mlsanity.com/paper/2111.12772/) (2021)
- [IKEA Furniture Assembly Environment for Long-Horizon Complex Manipulation Tasks](https://mlsanity.com/paper/1911.07246/) (2019)
- [Generative 3D Part Assembly via Dynamic Graph Learning](https://mlsanity.com/paper/2006.07793/) (2020)
- [Equivariant Descriptor Fields: SE(3)-Equivariant Energy-Based Models for End-to-End Visual Robotic Manipulation Learning](https://mlsanity.com/paper/2206.08321/) (2022)
- [Learning 3D Part Assembly from a Single Image](https://mlsanity.com/paper/2003.09754/) (2020)
- [Form2Fit: Learning Shape Priors for Generalizable Assembly from Disassembly](https://mlsanity.com/paper/1910.13675/) (2019)
- [SE(3)-Equivariant Relational Rearrangement with Neural Descriptor Fields](https://mlsanity.com/paper/2211.09786/) (2022)
- [Leveraging SE(3) Equivariance for Self-Supervised Category-Level Object Pose Estimation](https://mlsanity.com/paper/2111.00190/) (2021)
- [Neural Shape Mating: Self-Supervised Object Assembly with Adversarial Shape Priors](https://mlsanity.com/paper/2205.14886/) (2022)
- [Quaternion Equivariant Capsule Networks for 3D Point Clouds](https://mlsanity.com/paper/1912.12098/) (2019)
- [Breaking Bad: A Dataset for Geometric Fracture and Reassembly](https://mlsanity.com/paper/2210.11463/) (2022)
- [RGL-NET: A Recurrent Graph Learning framework for Progressive Part Assembly](https://mlsanity.com/paper/2107.12859/) (2021)
- [COALESCE: Component Assembly by Learning to Synthesize Connections](https://mlsanity.com/paper/2008.01936/) (2020)
- [USEEK: Unsupervised SE(3)-Equivariant 3D Keypoints for Generalizable Manipulation](https://mlsanity.com/paper/2209.13864/) (2022)
- [Self-Supervised Category-Level Articulated Object Pose Estimation with Part-Level SE(3) Equivariance](https://mlsanity.com/paper/2302.14268/) (2023)
- [Rotationally Equivariant 3D Object Detection](https://mlsanity.com/paper/2204.13630/) (2022)
- [3D Equivariant Graph Implicit Functions](https://mlsanity.com/paper/2203.17178/) (2022)
- [Shape-Pose Disentanglement using SE(3)-equivariant Vector Neurons](https://mlsanity.com/paper/2204.01159/) (2022)
- [Coarse-to-Fine Point Cloud Registration with SE(3)-Equivariant Representations](https://mlsanity.com/paper/2210.02045/) (2022)
- [Scene Editing as Teleoperation: A Case Study in 6DoF Kit Assembly](https://mlsanity.com/paper/2110.04450/) (2021)

## Built on by (most cited first, 28 of 28)

- [DiffAssemble: A Unified Graph-Diffusion Model for 2D and 3D Reassembly](https://mlsanity.com/paper/2402.19302/) (2024)
- [ET-SEED: Efficient Trajectory-Level SE(3) Equivariant Diffusion Policy](https://mlsanity.com/paper/2411.03990/) (2024)
- [Seek-CAD: A Self-refined Generative Modeling for 3D Parametric CAD Using Local Inference via DeepSeek](https://mlsanity.com/paper/2505.17702/) (2025)
- [Re-assembling the past: The RePAIR dataset and benchmark for real world 2D and 3D puzzle solving](https://mlsanity.com/paper/2410.24010/) (2024)
- [PuzzleFusion++: Auto-agglomerative 3D Fracture Assembly by Denoise and Verify](https://mlsanity.com/paper/2406.00259/) (2024)
- [Scalable Geometric Fracture Assembly via Co-creation Space among Assemblers](https://mlsanity.com/paper/2312.12340/) (2023)
- [Manual2Skill: Learning to Read Manuals and Acquire Robotic Skills for Furniture Assembly Using Vision-Language Models](https://mlsanity.com/paper/2502.10090/) (2025)
- [Rectified Point Flow: Generic Point Cloud Pose Estimation](https://mlsanity.com/paper/2506.05282/) (2025)
- [GARF: Learning Generalizable 3D Reassembly for Real-World Fractures](https://mlsanity.com/paper/2504.05400/) (2025)
- [Two by Two: Learning Multi-Task Pairwise Objects Assembly for Generalizable Robot Manipulation](https://mlsanity.com/paper/2504.06961/) (2025)
- [SPAFormer: Sequential 3D Part Assembly with Transformers](https://mlsanity.com/paper/2403.05874/) (2024)
- [3D Geometric Shape Assembly via Efficient Point Cloud Matching](https://mlsanity.com/paper/2407.10542/) (2024)
- [A Survey on Computational Solutions for Reconstructing Complete Objects by Reassembling Their Fractured Parts](https://mlsanity.com/paper/2410.14770/) (2024)
- [EqvAfford: SE(3) Equivariance for Point-Level Affordance Learning](https://mlsanity.com/paper/2408.01953/) (2024)
- [Jigsaw++: Imagining Complete Shape Priors for Object Reassembly](https://mlsanity.com/paper/2410.11816/) (2024)
- [BiAssemble: Learning Collaborative Affordance for Bimanual Geometric Assembly](https://mlsanity.com/paper/2506.06221/) (2025)
- [Manual2Skill++: Connector-Aware General Robotic Assembly from Instruction Manuals via Vision-Language Models](https://mlsanity.com/paper/2510.16344/) (2025)
- [SE(3)-bi-equivariant Transformers for Point Cloud Assembly](https://mlsanity.com/paper/2407.09167/) (2024)
- [TORA: Topological Representation Alignment for 3D Shape Assembly](https://mlsanity.com/paper/2604.04050/) (2026)
- [TreeSBA: Tree-Transformer for Self-Supervised Sequential Brick Assembly](https://mlsanity.com/paper/2407.15648/) (2024)
- [Geometric Point Attention Transformer for 3D Shape Reassembly](https://mlsanity.com/paper/2411.17788/) (2024)
- [Combinative Matching for Geometric Shape Assembly](https://mlsanity.com/paper/2508.09780/) (2025)
- [Equivariant Flow Matching for Point Cloud Assembly](https://mlsanity.com/paper/2505.21539/) (2025)
- [SARe: Structure-Aware Generative 3D Fragment Reassembly](https://mlsanity.com/paper/2603.21611/) (2026)
- [Transparent Fragments Contour Estimation via Visual-Tactile Fusion for Autonomous Reassembly](https://mlsanity.com/paper/2603.20290/) (2026)
- [LAMP: Lift Image-Editing as General 3D Priors for Open-world Manipulation](https://mlsanity.com/paper/2604.08475/) (2026)
- [PVRA: A Pointwise Key-point Voting Framework for Robotic Assembly](https://mlsanity.com/paper/2608.19968/) (2026)
- [LLM2R: From Large Language Models to 3D Reassemblers](https://mlsanity.com/paper/2610.18604/) (2026)

## Abstract (arXiv)

Shape assembly aims to reassemble parts (or fragments) into a complete object, which is a common task in our daily life. Different from the semantic part assembly (e.g., assembling a chair's semantic parts like legs into a whole chair), geometric part assembly (e.g., assembling bowl fragments into a complete bowl) is an emerging task in computer vision and robotics. Instead of semantic information, this task focuses on geometric information of parts. As the both geometric and pose space of fractured parts are exceptionally large, shape pose disentanglement of part representations is beneficial to geometric shape assembly. In our paper, we propose to leverage SE(3) equivariance for such shape pose disentanglement. Moreover, while previous works in vision and robotics only consider SE(3) equivariance for the representations of single objects, we move a step forward and propose leveraging SE(3) equivariance for representations considering multi-part correlations, which further boosts the performance of the multi-part assembly. Experiments demonstrate the significance of SE(3) equivariance and our proposed method for geometric shape assembly. Project page: https://crtie.github.io/SE-3-part-assembly/