# Fix the Match, Not the Solver: STITCH -- a Self-Supervised Compatibility Measure for Eroded Visual Puzzles

arXiv: https://arxiv.org/abs/2610.13176 · HTML: https://mlsanity.com/paper/2610.13176/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 53 references that are in the map.
- First posted on arXiv 9 Oct 2026.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Domain Generalization by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1903.06864/) (2019)
- [DiffAssemble: A Unified Graph-Diffusion Model for 2D and 3D Reassembly](https://mlsanity.com/paper/2402.19302/) (2024)
- [Deepzzle: Solving Visual Jigsaw Puzzles with Deep Learning andShortest Path Optimization](https://mlsanity.com/paper/2005.12548/) (2020)
- [Re-assembling the past: The RePAIR dataset and benchmark for real world 2D and 3D puzzle solving](https://mlsanity.com/paper/2410.24010/) (2024)
- [Solving Jigsaw Puzzles with Eroded Boundaries](https://mlsanity.com/paper/1912.00755/) (2019)
- [A Genetic Algorithm-Based Solver for Very Large Jigsaw Puzzles](https://mlsanity.com/paper/1711.06769/) (2017)
- [Solving Masked Jigsaw Puzzles with Diffusion Vision Transformers](https://mlsanity.com/paper/2404.07292/) (2024)
- [Solving Jigsaw Puzzles with Linear Programming](https://mlsanity.com/paper/1511.04472/) (2015)
- [Pictorial and apictorial polygonal jigsaw puzzles from arbitrary number of crossing cuts](https://mlsanity.com/paper/2008.07644/) (2020)
- [GANzzle: Reframing jigsaw puzzle solving as a retrieval task using a generative mental image](https://mlsanity.com/paper/2207.05634/) (2022)
- [Solving Jigsaw Puzzles By the Graph Connection Laplacian](https://mlsanity.com/paper/1811.03188/) (2018)
- [Multi-Phase Relaxation Labeling for Square Jigsaw Puzzle Solving](https://mlsanity.com/paper/2303.14793/) (2023)
- [DNN-Buddies: A Deep Neural Network-Based Estimation Metric for the Jigsaw Puzzle Problem](https://mlsanity.com/paper/1711.08762/) (2017)
- [Pairwise Alignment & Compatibility for Arbitrarily Irregular Image Fragments](https://mlsanity.com/paper/2507.09767/) (2025)
- [ERL-MPP: Evolutionary Reinforcement Learning with Multi-head Puzzle Perception for Solving Large-scale Jigsaw Puzzles of Eroded Gaps](https://mlsanity.com/paper/2504.09608/) (2025)
- [TEN: Twin Embedding Networks for the Jigsaw Puzzle Problem with Eroded Boundaries](https://mlsanity.com/paper/2203.06488/) (2022)
- [VLHSA: Vision-Language Hierarchical Semantic Alignment for Jigsaw Puzzle Solving with Eroded Gaps](https://mlsanity.com/paper/2509.25202/) (2025)
- [The Missing GAP: From Solving Square Jigsaw Puzzles to Handling Real World Archaeological Fragments](https://mlsanity.com/paper/2605.12077/) (2026)
- [Relaxation Labeling Meets GANs: Solving Jigsaw Puzzles with Missing Borders](https://mlsanity.com/paper/2203.14428/) (2022)
- [Edge2Vec: A High Quality Embedding for the Jigsaw Puzzle Problem](https://mlsanity.com/paper/2211.07771/) (2022)

## Abstract (arXiv)

Unlike the clean and pixel-aligned complete pieces of standard jigsaw benchmarks, real-world puzzle fragments lose their boundary regions to erosion, destroying the signal that classical compatibility measures (CMs) rely on. We first re-establish, via oracle experiments, that the CM, not the solver, is the performance bottleneck: with a perfect CM, even a simple greedy solver achieves perfect reconstruction. This motivates STITCH, a self-supervised CM that sidesteps missing boundaries by comparing frozen ViT patch tokens from piece interiors via a lightweight cross-attention head (~249K trainable parameters), trained without manual annotation. Plugged into unchanged compatibility-based solvers, STITCH reaches .992/.954 Abs/Perf on JPwLEG-5 (+.629 Perf over the best prior method), nearly saturating the benchmark. On a new ImageNet 7x7 benchmark, it outperforms all alternatives, including inpaint-then-match pipelines built on modern generative models. Importantly, performance remains robust across erosion levels and generalizes to puzzles with over 40% missing pieces without retraining. Trained on the recent GAP datasets of irregular, heavily eroded fragments, the same CM recipe enables compatibility-based solvers to surpass recent learning-based solvers on GAP-3 and GAP-5. Since this leap in performance is achieved by a pluggable CM module, the takeaway is clear: fix the match, not the solver.