# Jigsaw: Learning to Assemble Multiple Fractured Objects

arXiv: https://arxiv.org/abs/2305.17975 · HTML: https://mlsanity.com/paper/2305.17975/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 41 references that are in the map.
- First posted on arXiv 29 May 2023.

## Builds on (most cited first, 25 of 25)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [PREDATOR: Registration of 3D Point Clouds with Low Overlap](https://mlsanity.com/paper/2011.13005/) (2020)
- [Geometric Transformer for Fast and Robust Point Cloud Registration](https://mlsanity.com/paper/2202.06688/) (2022)
- [PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes](https://mlsanity.com/paper/1911.10949/) (2019)
- [Learning Combinatorial Embedding Networks for Deep Graph Matching](https://mlsanity.com/paper/1904.00597/) (2019)
- [Neural Graph Matching Network: Learning Lawler's Quadratic Assignment Problem with Extension to Hypergraph and Multiple-graph Matching](https://mlsanity.com/paper/1911.11308/) (2019)
- [Deep Graph Matching via Blackbox Differentiation of Combinatorial Solvers](https://mlsanity.com/paper/2003.11657/) (2020)
- [JoinABLe: Learning Bottom-up Assembly of Parametric CAD Joints](https://mlsanity.com/paper/2111.12772/) (2021)
- [Generative 3D Part Assembly via Dynamic Graph Learning](https://mlsanity.com/paper/2006.07793/) (2020)
- [Learning 3D Part Assembly from a Single Image](https://mlsanity.com/paper/2003.09754/) (2020)
- [Shonan Rotation Averaging: Global Optimality by Surfing SO(p)n](https://mlsanity.com/paper/2008.02737/) (2020)
- [Neural Shape Mating: Self-Supervised Object Assembly with Adversarial Shape Priors](https://mlsanity.com/paper/2205.14886/) (2022)
- [Breaking Bad: A Dataset for Geometric Fracture and Reassembly](https://mlsanity.com/paper/2210.11463/) (2022)
- [RGL-NET: A Recurrent Graph Learning framework for Progressive Part Assembly](https://mlsanity.com/paper/2107.12859/) (2021)
- [Composite Shape Modeling via Latent Space Factorization](https://mlsanity.com/paper/1901.02968/) (2019)
- [COALESCE: Component Assembly by Learning to Synthesize Connections](https://mlsanity.com/paper/2008.01936/) (2020)
- [Point Cloud Registration using Representative Overlapping Points](https://mlsanity.com/paper/2107.02583/) (2021)
- [Learning Universe Model for Partial Matching Networks over Multiple Graphs](https://mlsanity.com/paper/2210.10374/) (2022)

## Built on by (most cited first, 29 of 29)

- [Re-assembling the past: The RePAIR dataset and benchmark for real world 2D and 3D puzzle solving](https://mlsanity.com/paper/2410.24010/) (2024)
- [GaussianArt: Unified Modeling of Geometry and Motion for Articulated Objects](https://mlsanity.com/paper/2508.14891/) (2025)
- [PuzzleFusion++: Auto-agglomerative 3D Fracture Assembly by Denoise and Verify](https://mlsanity.com/paper/2406.00259/) (2024)
- [Rectified Point Flow: Generic Point Cloud Pose Estimation](https://mlsanity.com/paper/2506.05282/) (2025)
- [GARF: Learning Generalizable 3D Reassembly for Real-World Fractures](https://mlsanity.com/paper/2504.05400/) (2025)
- [Two by Two: Learning Multi-Task Pairwise Objects Assembly for Generalizable Robot Manipulation](https://mlsanity.com/paper/2504.06961/) (2025)
- [SPAFormer: Sequential 3D Part Assembly with Transformers](https://mlsanity.com/paper/2403.05874/) (2024)
- [Assembler: Scalable 3D Part Assembly via Anchor Point Diffusion](https://mlsanity.com/paper/2506.17074/) (2025)
- [3D Geometric Shape Assembly via Efficient Point Cloud Matching](https://mlsanity.com/paper/2407.10542/) (2024)
- [A Survey on Computational Solutions for Reconstructing Complete Objects by Reassembling Their Fractured Parts](https://mlsanity.com/paper/2410.14770/) (2024)
- [Jigsaw++: Imagining Complete Shape Priors for Object Reassembly](https://mlsanity.com/paper/2410.11816/) (2024)
- [Effective Protein-Protein Interaction Exploration with PPIretrieval](https://mlsanity.com/paper/2402.03675/) (2024)
- [BiAssemble: Learning Collaborative Affordance for Bimanual Geometric Assembly](https://mlsanity.com/paper/2506.06221/) (2025)
- [GarmageNet: A Multimodal Generative Framework for Sewing Pattern Design and Generic Garment Modeling](https://mlsanity.com/paper/2504.01483/) (2025)
- [TORA: Topological Representation Alignment for 3D Shape Assembly](https://mlsanity.com/paper/2604.04050/) (2026)
- [Geometric Point Attention Transformer for 3D Shape Reassembly](https://mlsanity.com/paper/2411.17788/) (2024)
- [Combinative Matching for Geometric Shape Assembly](https://mlsanity.com/paper/2508.09780/) (2025)
- [Interact3D: Compositional 3D Generation of Interactive Objects](https://mlsanity.com/paper/2603.16085/) (2026)
- [Equivariant Flow Matching for Point Cloud Assembly](https://mlsanity.com/paper/2505.21539/) (2025)
- [E-M3RF: An Equivariant Multimodal 3D Re-assembly Framework](https://mlsanity.com/paper/2511.21422/) (2025)
- [Learning Sewing Patterns via Latent Flow Matching of Implicit Fields](https://mlsanity.com/paper/2601.17740/) (2026)
- [CRAG: Can 3D Generative Models Help 3D Assembly?](https://mlsanity.com/paper/2602.22629/) (2026)
- [AutoSew: A Geometric Approach to Stitching Prediction with Graph Neural Networks](https://mlsanity.com/paper/2602.22052/) (2026)
- [Transparent Fragments Contour Estimation via Visual-Tactile Fusion for Autonomous Reassembly](https://mlsanity.com/paper/2603.20290/) (2026)
- [LAMP: Lift Image-Editing as General 3D Priors for Open-world Manipulation](https://mlsanity.com/paper/2604.08475/) (2026)
- [AssemblyWorld: Rethinking 3D Assembly with General-Purpose Agents](https://mlsanity.com/paper/2609.40353/) (2026)
- [Automatic assembly modeling through attention-based assembly feature mate prediction and assembly feature pattern recognition](https://mlsanity.com/paper/2610.14633/) (2026)
- [LLM2R: From Large Language Models to 3D Reassemblers](https://mlsanity.com/paper/2610.18604/) (2026)
- [PhySGA: A Physics-Space Grounded Agentic Framework for Generalizable and Interactive 3D Composition](https://mlsanity.com/paper/2610.20560/) (2026)

## Abstract (arXiv)

Automated assembly of 3D fractures is essential in orthopedics, archaeology, and our daily life. This paper presents Jigsaw, a novel framework for assembling physically broken 3D objects from multiple pieces. Our approach leverages hierarchical features of global and local geometry to match and align the fracture surfaces. Our framework consists of four components: (1) front-end point feature extractor with attention layers, (2) surface segmentation to separate fracture and original parts, (3) multi-parts matching to find correspondences among fracture surface points, and (4) robust global alignment to recover the global poses of the pieces. We show how to jointly learn segmentation and matching and seamlessly integrate feature matching and rigidity constraints. We evaluate Jigsaw on the Breaking Bad dataset and achieve superior performance compared to state-of-the-art methods. Our method also generalizes well to diverse fracture modes, objects, and unseen instances. To the best of our knowledge, this is the first learning-based method designed specifically for 3D fracture assembly over multiple pieces. Our code is available at https://jiaxin-lu.github.io/Jigsaw/.